Note: This is test shard 6 of 8.
[==========] Running 9 tests from 5 test suites.
[----------] Global test environment set-up.
[----------] 5 tests from AdminCliTest
[ RUN ] AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250809 14:08:16.997313 31831 test_util.cc:276] Using random seed: 953510834
I20250809 14:08:17.001974 31831 ts_itest-base.cc:115] Starting cluster with:
I20250809 14:08:17.002018 31831 ts_itest-base.cc:116] --------------
I20250809 14:08:17.002039 31831 ts_itest-base.cc:117] 4 tablet servers
I20250809 14:08:17.002058 31831 ts_itest-base.cc:118] 3 replicas per TS
I20250809 14:08:17.002075 31831 ts_itest-base.cc:119] --------------
2025-08-09T14:08:17Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-09T14:08:17Z Disabled control of system clock
I20250809 14:08:17.009425 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:33959
--webserver_interface=127.31.21.254
--webserver_port=0
--builtin_ntp_servers=127.31.21.212:37333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.21.254:33959 with env {}
W20250809 14:08:17.080781 31843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:17.080960 31843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:17.080979 31843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:17.082340 31843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250809 14:08:17.082377 31843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:17.082391 31843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250809 14:08:17.082401 31843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250809 14:08:17.083854 31843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:37333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.21.254:33959
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:33959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.21.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.31843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:17.084066 31843 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:17.084276 31843 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:17.086504 31851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:17.086619 31848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:17.086508 31849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:17.086793 31843 server_base.cc:1047] running on GCE node
I20250809 14:08:17.086985 31843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:17.087188 31843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:17.088322 31843 hybrid_clock.cc:648] HybridClock initialized: now 1754748497088304 us; error 31 us; skew 500 ppm
I20250809 14:08:17.089421 31843 webserver.cc:489] Webserver started at http://127.31.21.254:46441/ using document root <none> and password file <none>
I20250809 14:08:17.089622 31843 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:17.089666 31843 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:17.089766 31843 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:17.090566 31843 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "68c1a54aa076402181b59b73b4847e60"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.090852 31843 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "68c1a54aa076402181b59b73b4847e60"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.092026 31843 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250809 14:08:17.092708 31857 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.092877 31843 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:17.092970 31843 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
uuid: "68c1a54aa076402181b59b73b4847e60"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.093030 31843 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:17.105602 31843 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:17.105832 31843 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:17.105937 31843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:17.109477 31843 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.254:33959
I20250809 14:08:17.109510 31909 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.254:33959 every 8 connection(s)
I20250809 14:08:17.109809 31843 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250809 14:08:17.110363 31910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:17.112517 31910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60: Bootstrap starting.
I20250809 14:08:17.112871 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 31843
I20250809 14:08:17.112984 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250809 14:08:17.113298 31910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:17.113622 31910 log.cc:826] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:17.114712 31910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60: No bootstrap required, opened a new log
I20250809 14:08:17.116314 31910 raft_consensus.cc:357] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c1a54aa076402181b59b73b4847e60" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 33959 } }
I20250809 14:08:17.116432 31910 raft_consensus.cc:383] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:17.116462 31910 raft_consensus.cc:738] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c1a54aa076402181b59b73b4847e60, State: Initialized, Role: FOLLOWER
I20250809 14:08:17.116588 31910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c1a54aa076402181b59b73b4847e60" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 33959 } }
I20250809 14:08:17.116657 31910 raft_consensus.cc:397] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:17.116703 31910 raft_consensus.cc:491] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:17.116750 31910 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:17.117358 31910 raft_consensus.cc:513] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c1a54aa076402181b59b73b4847e60" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 33959 } }
I20250809 14:08:17.117441 31910 leader_election.cc:304] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68c1a54aa076402181b59b73b4847e60; no voters:
I20250809 14:08:17.117580 31910 leader_election.cc:290] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:17.117624 31915 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:17.117756 31915 raft_consensus.cc:695] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 1 LEADER]: Becoming Leader. State: Replica: 68c1a54aa076402181b59b73b4847e60, State: Running, Role: LEADER
I20250809 14:08:17.117810 31910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [sys.catalog]: configured and running, proceeding with master startup.
I20250809 14:08:17.117874 31915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c1a54aa076402181b59b73b4847e60" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 33959 } }
I20250809 14:08:17.118150 31915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68c1a54aa076402181b59b73b4847e60. Latest consensus state: current_term: 1 leader_uuid: "68c1a54aa076402181b59b73b4847e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c1a54aa076402181b59b73b4847e60" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 33959 } } }
I20250809 14:08:17.118219 31915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:17.118161 31916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68c1a54aa076402181b59b73b4847e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c1a54aa076402181b59b73b4847e60" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 33959 } } }
I20250809 14:08:17.118448 31916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:17.118444 31919 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250809 14:08:17.119063 31919 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250809 14:08:17.120612 31919 catalog_manager.cc:1349] Generated new cluster ID: 81c4a7142151457fb3f0543863312210
I20250809 14:08:17.120662 31919 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250809 14:08:17.127784 31919 catalog_manager.cc:1372] Generated new certificate authority record
I20250809 14:08:17.128263 31919 catalog_manager.cc:1506] Loading token signing keys...
I20250809 14:08:17.133451 31919 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60: Generated new TSK 0
I20250809 14:08:17.133601 31919 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250809 14:08:17.135087 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.193:0
--local_ip_for_outbound_sockets=127.31.21.193
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:33959
--builtin_ntp_servers=127.31.21.212:37333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250809 14:08:17.208194 31934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:17.208365 31934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:17.208398 31934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:17.209743 31934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:17.209795 31934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.193
I20250809 14:08:17.211158 31934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:37333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:33959
--never_fsync=true
--heap_profile_path=/tmp/kudu.31934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.193
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:17.211344 31934 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:17.211586 31934 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:17.213953 31942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:17.213964 31940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:17.214020 31934 server_base.cc:1047] running on GCE node
W20250809 14:08:17.213964 31939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:17.214332 31934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:17.214511 31934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:17.215633 31934 hybrid_clock.cc:648] HybridClock initialized: now 1754748497215622 us; error 29 us; skew 500 ppm
I20250809 14:08:17.216665 31934 webserver.cc:489] Webserver started at http://127.31.21.193:34107/ using document root <none> and password file <none>
I20250809 14:08:17.216869 31934 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:17.216928 31934 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:17.217037 31934 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:17.217890 31934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "178860eeed854ca48128d5588ecc77c4"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.218179 31934 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "178860eeed854ca48128d5588ecc77c4"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.219389 31934 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250809 14:08:17.220032 31948 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.220196 31934 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:17.220261 31934 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "178860eeed854ca48128d5588ecc77c4"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.220316 31934 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:17.239233 31934 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:17.239461 31934 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:17.239565 31934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:17.239754 31934 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:17.240069 31934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:17.240103 31934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.240136 31934 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:17.240170 31934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.245816 31934 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.193:46367
I20250809 14:08:17.245862 32061 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.193:46367 every 8 connection(s)
I20250809 14:08:17.246126 31934 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250809 14:08:17.249014 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 31934
I20250809 14:08:17.249107 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250809 14:08:17.250253 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.194:0
--local_ip_for_outbound_sockets=127.31.21.194
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:33959
--builtin_ntp_servers=127.31.21.212:37333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250809 14:08:17.250819 32062 heartbeater.cc:344] Connected to a master server at 127.31.21.254:33959
I20250809 14:08:17.250910 32062 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:17.251112 32062 heartbeater.cc:507] Master 127.31.21.254:33959 requested a full tablet report, sending...
I20250809 14:08:17.251623 31874 ts_manager.cc:194] Registered new tserver with Master: 178860eeed854ca48128d5588ecc77c4 (127.31.21.193:46367)
I20250809 14:08:17.252283 31874 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.193:52119
W20250809 14:08:17.328142 32065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:17.328303 32065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:17.328322 32065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:17.329645 32065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:17.329694 32065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.194
I20250809 14:08:17.331096 32065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:37333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:33959
--never_fsync=true
--heap_profile_path=/tmp/kudu.32065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.194
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:17.331286 32065 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:17.331504 32065 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:17.333829 32073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:17.333851 32071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:17.333863 32070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:17.334111 32065 server_base.cc:1047] running on GCE node
I20250809 14:08:17.334255 32065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:17.334452 32065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:17.335575 32065 hybrid_clock.cc:648] HybridClock initialized: now 1754748497335560 us; error 26 us; skew 500 ppm
I20250809 14:08:17.336589 32065 webserver.cc:489] Webserver started at http://127.31.21.194:43961/ using document root <none> and password file <none>
I20250809 14:08:17.336758 32065 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:17.336800 32065 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:17.336925 32065 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:17.337687 32065 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "d8f343363bd4454fa9122b98174782a0"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.337962 32065 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "d8f343363bd4454fa9122b98174782a0"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.338990 32065 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250809 14:08:17.339557 32079 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.339704 32065 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:17.339768 32065 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "d8f343363bd4454fa9122b98174782a0"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.339823 32065 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:17.369230 32065 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:17.369488 32065 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:17.369604 32065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:17.369822 32065 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:17.370143 32065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:17.370175 32065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.370213 32065 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:17.370237 32065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.375540 32065 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.194:36835
I20250809 14:08:17.375597 32192 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.194:36835 every 8 connection(s)
I20250809 14:08:17.375866 32065 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250809 14:08:17.379900 32193 heartbeater.cc:344] Connected to a master server at 127.31.21.254:33959
I20250809 14:08:17.380002 32193 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:17.380187 32193 heartbeater.cc:507] Master 127.31.21.254:33959 requested a full tablet report, sending...
I20250809 14:08:17.380541 31874 ts_manager.cc:194] Registered new tserver with Master: d8f343363bd4454fa9122b98174782a0 (127.31.21.194:36835)
I20250809 14:08:17.380918 31874 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.194:45159
I20250809 14:08:17.385010 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 32065
I20250809 14:08:17.385080 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250809 14:08:17.385937 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.195:0
--local_ip_for_outbound_sockets=127.31.21.195
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:33959
--builtin_ntp_servers=127.31.21.212:37333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250809 14:08:17.458030 32196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:17.458194 32196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:17.458212 32196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:17.459578 32196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:17.459630 32196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.195
I20250809 14:08:17.461053 32196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:37333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:33959
--never_fsync=true
--heap_profile_path=/tmp/kudu.32196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.195
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:17.461263 32196 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:17.461465 32196 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:17.464061 32201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:17.464102 32196 server_base.cc:1047] running on GCE node
W20250809 14:08:17.464058 32204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:17.464058 32202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:17.464411 32196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:17.464581 32196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:17.465694 32196 hybrid_clock.cc:648] HybridClock initialized: now 1754748497465685 us; error 26 us; skew 500 ppm
I20250809 14:08:17.466717 32196 webserver.cc:489] Webserver started at http://127.31.21.195:44545/ using document root <none> and password file <none>
I20250809 14:08:17.466905 32196 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:17.466944 32196 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:17.467046 32196 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:17.467862 32196 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "5e59681166ff4bbdb87acea5800e3ae9"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.468171 32196 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "5e59681166ff4bbdb87acea5800e3ae9"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.469318 32196 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250809 14:08:17.469981 32210 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.470132 32196 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250809 14:08:17.470202 32196 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "5e59681166ff4bbdb87acea5800e3ae9"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.470263 32196 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:17.482553 32196 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:17.482798 32196 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:17.482911 32196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:17.483115 32196 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:17.483407 32196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:17.483438 32196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.483472 32196 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:17.483501 32196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.489264 32196 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.195:42179
I20250809 14:08:17.489333 32323 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.195:42179 every 8 connection(s)
I20250809 14:08:17.489586 32196 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250809 14:08:17.493733 32324 heartbeater.cc:344] Connected to a master server at 127.31.21.254:33959
I20250809 14:08:17.493814 32324 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:17.493976 32324 heartbeater.cc:507] Master 127.31.21.254:33959 requested a full tablet report, sending...
I20250809 14:08:17.494292 31874 ts_manager.cc:194] Registered new tserver with Master: 5e59681166ff4bbdb87acea5800e3ae9 (127.31.21.195:42179)
I20250809 14:08:17.494606 31874 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.195:46043
I20250809 14:08:17.499780 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 32196
I20250809 14:08:17.499852 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250809 14:08:17.500742 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.196:0
--local_ip_for_outbound_sockets=127.31.21.196
--webserver_interface=127.31.21.196
--webserver_port=0
--tserver_master_addrs=127.31.21.254:33959
--builtin_ntp_servers=127.31.21.212:37333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250809 14:08:17.574426 32327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:17.574570 32327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:17.574587 32327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:17.575773 32327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:17.575815 32327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.196
I20250809 14:08:17.577126 32327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:37333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.31.21.196
--webserver_port=0
--tserver_master_addrs=127.31.21.254:33959
--never_fsync=true
--heap_profile_path=/tmp/kudu.32327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.196
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:17.577292 32327 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:17.577463 32327 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:17.579666 32332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:17.579696 32333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:17.579710 32335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:17.579842 32327 server_base.cc:1047] running on GCE node
I20250809 14:08:17.579983 32327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:17.580179 32327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:17.581336 32327 hybrid_clock.cc:648] HybridClock initialized: now 1754748497581312 us; error 27 us; skew 500 ppm
I20250809 14:08:17.582401 32327 webserver.cc:489] Webserver started at http://127.31.21.196:34355/ using document root <none> and password file <none>
I20250809 14:08:17.582581 32327 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:17.582638 32327 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:17.582757 32327 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:17.583545 32327 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "901a5a1c50f145e69dbc4ea9757d607b"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.583819 32327 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "901a5a1c50f145e69dbc4ea9757d607b"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.584996 32327 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20250809 14:08:17.585660 32341 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.585800 32327 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.585867 32327 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "901a5a1c50f145e69dbc4ea9757d607b"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:17.585924 32327 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:17.598490 32327 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:17.598717 32327 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:17.598827 32327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:17.599017 32327 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:17.599304 32327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:17.599336 32327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.599368 32327 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:17.599397 32327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:17.604820 32327 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.196:40077
I20250809 14:08:17.604920 32454 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.196:40077 every 8 connection(s)
I20250809 14:08:17.605173 32327 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20250809 14:08:17.609048 32455 heartbeater.cc:344] Connected to a master server at 127.31.21.254:33959
I20250809 14:08:17.609130 32455 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:17.609289 32455 heartbeater.cc:507] Master 127.31.21.254:33959 requested a full tablet report, sending...
I20250809 14:08:17.609570 31874 ts_manager.cc:194] Registered new tserver with Master: 901a5a1c50f145e69dbc4ea9757d607b (127.31.21.196:40077)
I20250809 14:08:17.609905 31874 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.196:36481
I20250809 14:08:17.614739 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 32327
I20250809 14:08:17.614835 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20250809 14:08:17.615839 31831 external_mini_cluster.cc:949] 4 TS(s) registered with all masters
I20250809 14:08:17.621729 31874 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55628:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20250809 14:08:17.628558 31996 tablet_service.cc:1468] Processing CreateTablet for tablet 53e7161edc9d4aecb6672d0c7d3484eb (DEFAULT_TABLE table=TestTable [id=360632eed4e84fac85aff7f5c23600de]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:17.628943 31996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e7161edc9d4aecb6672d0c7d3484eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:17.628903 32389 tablet_service.cc:1468] Processing CreateTablet for tablet 53e7161edc9d4aecb6672d0c7d3484eb (DEFAULT_TABLE table=TestTable [id=360632eed4e84fac85aff7f5c23600de]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:17.628901 32127 tablet_service.cc:1468] Processing CreateTablet for tablet 53e7161edc9d4aecb6672d0c7d3484eb (DEFAULT_TABLE table=TestTable [id=360632eed4e84fac85aff7f5c23600de]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:17.629189 32389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e7161edc9d4aecb6672d0c7d3484eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:17.629199 32127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e7161edc9d4aecb6672d0c7d3484eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:17.630914 32469 tablet_bootstrap.cc:492] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4: Bootstrap starting.
I20250809 14:08:17.631388 32469 tablet_bootstrap.cc:654] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:17.631580 32469 log.cc:826] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:17.632048 32471 tablet_bootstrap.cc:492] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b: Bootstrap starting.
I20250809 14:08:17.632048 32470 tablet_bootstrap.cc:492] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0: Bootstrap starting.
I20250809 14:08:17.632205 32469 tablet_bootstrap.cc:492] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4: No bootstrap required, opened a new log
I20250809 14:08:17.632267 32469 ts_tablet_manager.cc:1397] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:17.632656 32471 tablet_bootstrap.cc:654] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:17.632656 32470 tablet_bootstrap.cc:654] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:17.632949 32470 log.cc:826] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:17.632949 32471 log.cc:826] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:17.633531 32470 tablet_bootstrap.cc:492] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0: No bootstrap required, opened a new log
I20250809 14:08:17.633409 32469 raft_consensus.cc:357] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } }
I20250809 14:08:17.633586 32469 raft_consensus.cc:383] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:17.633589 32470 ts_tablet_manager.cc:1397] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:17.633659 32469 raft_consensus.cc:738] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 178860eeed854ca48128d5588ecc77c4, State: Initialized, Role: FOLLOWER
I20250809 14:08:17.633699 32471 tablet_bootstrap.cc:492] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b: No bootstrap required, opened a new log
I20250809 14:08:17.633760 32471 ts_tablet_manager.cc:1397] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:17.633775 32469 consensus_queue.cc:260] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } }
I20250809 14:08:17.634032 32469 ts_tablet_manager.cc:1428] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:17.634064 32062 heartbeater.cc:499] Master 127.31.21.254:33959 was elected leader, sending a full tablet report...
I20250809 14:08:17.634936 32471 raft_consensus.cc:357] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } }
I20250809 14:08:17.635032 32471 raft_consensus.cc:383] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:17.635059 32471 raft_consensus.cc:738] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 901a5a1c50f145e69dbc4ea9757d607b, State: Initialized, Role: FOLLOWER
I20250809 14:08:17.635149 32471 consensus_queue.cc:260] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } }
I20250809 14:08:17.635221 32470 raft_consensus.cc:357] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } }
I20250809 14:08:17.635326 32471 ts_tablet_manager.cc:1428] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:17.635342 32455 heartbeater.cc:499] Master 127.31.21.254:33959 was elected leader, sending a full tablet report...
I20250809 14:08:17.635339 32470 raft_consensus.cc:383] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:17.635370 32470 raft_consensus.cc:738] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f343363bd4454fa9122b98174782a0, State: Initialized, Role: FOLLOWER
I20250809 14:08:17.635439 32470 consensus_queue.cc:260] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } }
I20250809 14:08:17.635639 32470 ts_tablet_manager.cc:1428] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:17.635675 32193 heartbeater.cc:499] Master 127.31.21.254:33959 was elected leader, sending a full tablet report...
I20250809 14:08:17.679386 32476 raft_consensus.cc:491] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250809 14:08:17.679500 32476 raft_consensus.cc:513] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } }
I20250809 14:08:17.679826 32476 leader_election.cc:290] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8f343363bd4454fa9122b98174782a0 (127.31.21.194:36835), 178860eeed854ca48128d5588ecc77c4 (127.31.21.193:46367)
I20250809 14:08:17.682233 32147 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "53e7161edc9d4aecb6672d0c7d3484eb" candidate_uuid: "901a5a1c50f145e69dbc4ea9757d607b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f343363bd4454fa9122b98174782a0" is_pre_election: true
I20250809 14:08:17.682389 32147 raft_consensus.cc:2466] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 901a5a1c50f145e69dbc4ea9757d607b in term 0.
I20250809 14:08:17.682607 32343 leader_election.cc:304] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 901a5a1c50f145e69dbc4ea9757d607b, d8f343363bd4454fa9122b98174782a0; no voters:
I20250809 14:08:17.682735 32476 raft_consensus.cc:2802] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250809 14:08:17.682806 32476 raft_consensus.cc:491] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250809 14:08:17.682826 32476 raft_consensus.cc:3058] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:17.682837 32016 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "53e7161edc9d4aecb6672d0c7d3484eb" candidate_uuid: "901a5a1c50f145e69dbc4ea9757d607b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "178860eeed854ca48128d5588ecc77c4" is_pre_election: true
I20250809 14:08:17.683008 32016 raft_consensus.cc:2466] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 901a5a1c50f145e69dbc4ea9757d607b in term 0.
I20250809 14:08:17.687197 32476 raft_consensus.cc:513] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } }
I20250809 14:08:17.687355 32476 leader_election.cc:290] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [CANDIDATE]: Term 1 election: Requested vote from peers d8f343363bd4454fa9122b98174782a0 (127.31.21.194:36835), 178860eeed854ca48128d5588ecc77c4 (127.31.21.193:46367)
I20250809 14:08:17.687529 32016 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "53e7161edc9d4aecb6672d0c7d3484eb" candidate_uuid: "901a5a1c50f145e69dbc4ea9757d607b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "178860eeed854ca48128d5588ecc77c4"
I20250809 14:08:17.687529 32147 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "53e7161edc9d4aecb6672d0c7d3484eb" candidate_uuid: "901a5a1c50f145e69dbc4ea9757d607b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f343363bd4454fa9122b98174782a0"
I20250809 14:08:17.687613 32016 raft_consensus.cc:3058] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:17.687613 32147 raft_consensus.cc:3058] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:17.688436 32016 raft_consensus.cc:2466] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 901a5a1c50f145e69dbc4ea9757d607b in term 1.
I20250809 14:08:17.688436 32147 raft_consensus.cc:2466] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 901a5a1c50f145e69dbc4ea9757d607b in term 1.
I20250809 14:08:17.688639 32344 leader_election.cc:304] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 178860eeed854ca48128d5588ecc77c4, 901a5a1c50f145e69dbc4ea9757d607b; no voters:
I20250809 14:08:17.688727 32476 raft_consensus.cc:2802] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:17.688886 32476 raft_consensus.cc:695] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [term 1 LEADER]: Becoming Leader. State: Replica: 901a5a1c50f145e69dbc4ea9757d607b, State: Running, Role: LEADER
I20250809 14:08:17.688982 32476 consensus_queue.cc:237] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } }
I20250809 14:08:17.689627 31874 catalog_manager.cc:5582] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b reported cstate change: term changed from 0 to 1, leader changed from <none> to 901a5a1c50f145e69dbc4ea9757d607b (127.31.21.196). New cstate: current_term: 1 leader_uuid: "901a5a1c50f145e69dbc4ea9757d607b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } health_report { overall_health: UNKNOWN } } }
I20250809 14:08:17.701148 31831 external_mini_cluster.cc:949] 4 TS(s) registered with all masters
I20250809 14:08:17.701645 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 178860eeed854ca48128d5588ecc77c4 to finish bootstrapping
I20250809 14:08:17.704727 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver d8f343363bd4454fa9122b98174782a0 to finish bootstrapping
I20250809 14:08:17.707561 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 901a5a1c50f145e69dbc4ea9757d607b to finish bootstrapping
I20250809 14:08:17.710192 31831 kudu-admin-test.cc:709] Waiting for Master to see the current replicas...
I20250809 14:08:17.710625 31831 kudu-admin-test.cc:716] Tablet locations:
tablet_locations {
tablet_id: "53e7161edc9d4aecb6672d0c7d3484eb"
DEPRECATED_stale: false
partition {
partition_key_start: ""
partition_key_end: ""
}
interned_replicas {
ts_info_idx: 0
role: LEADER
}
interned_replicas {
ts_info_idx: 1
role: FOLLOWER
}
interned_replicas {
ts_info_idx: 2
role: FOLLOWER
}
}
ts_infos {
permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b"
rpc_addresses {
host: "127.31.21.196"
port: 40077
}
}
ts_infos {
permanent_uuid: "d8f343363bd4454fa9122b98174782a0"
rpc_addresses {
host: "127.31.21.194"
port: 36835
}
}
ts_infos {
permanent_uuid: "178860eeed854ca48128d5588ecc77c4"
rpc_addresses {
host: "127.31.21.193"
port: 46367
}
}
W20250809 14:08:17.746752 32063 tablet.cc:2378] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250809 14:08:17.855734 32456 tablet.cc:2378] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250809 14:08:17.876520 32194 tablet.cc:2378] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250809 14:08:18.118594 32476 consensus_queue.cc:1035] T 53e7161edc9d4aecb6672d0c7d3484eb P 901a5a1c50f145e69dbc4ea9757d607b [LEADER]: Connected to new peer: Peer: permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250809 14:08:18.121032 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 32327
W20250809 14:08:18.126004 32083 connection.cc:537] server connection from 127.31.21.196:59187 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250809 14:08:18.126374 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 31843
I20250809 14:08:18.133077 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:33959
--webserver_interface=127.31.21.254
--webserver_port=46441
--builtin_ntp_servers=127.31.21.212:37333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.21.254:33959 with env {}
W20250809 14:08:18.208629 32487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:18.208781 32487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:18.208810 32487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:18.210186 32487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250809 14:08:18.210228 32487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:18.210250 32487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250809 14:08:18.210268 32487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250809 14:08:18.211745 32487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:37333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.21.254:33959
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:33959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.21.254
--webserver_port=46441
--never_fsync=true
--heap_profile_path=/tmp/kudu.32487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:18.211966 32487 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:18.212165 32487 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:18.214387 32493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:18.214504 32495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:18.214586 32492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:18.214632 32487 server_base.cc:1047] running on GCE node
I20250809 14:08:18.214811 32487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:18.215009 32487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:18.216132 32487 hybrid_clock.cc:648] HybridClock initialized: now 1754748498216121 us; error 28 us; skew 500 ppm
I20250809 14:08:18.217293 32487 webserver.cc:489] Webserver started at http://127.31.21.254:46441/ using document root <none> and password file <none>
I20250809 14:08:18.217478 32487 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:18.217537 32487 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:18.218678 32487 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:18.219293 32501 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:18.219553 32487 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:18.219622 32487 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
uuid: "68c1a54aa076402181b59b73b4847e60"
format_stamp: "Formatted at 2025-08-09 14:08:17 on dist-test-slave-mstq"
I20250809 14:08:18.219853 32487 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:18.241169 32487 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:18.241433 32487 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:18.241539 32487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:18.245254 32487 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.254:33959
I20250809 14:08:18.245306 32553 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.254:33959 every 8 connection(s)
I20250809 14:08:18.245627 32487 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250809 14:08:18.246814 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 32487
I20250809 14:08:18.246923 31831 kudu-admin-test.cc:735] Forcing unsafe config change on tserver d8f343363bd4454fa9122b98174782a0
I20250809 14:08:18.246984 32554 sys_catalog.cc:263] Verifying existing consensus state
I20250809 14:08:18.247763 32554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60: Bootstrap starting.
I20250809 14:08:18.255224 32554 log.cc:826] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:18.256759 32554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=5 ignored=0} mutations{seen=2 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:18.257062 32554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60: Bootstrap complete.
I20250809 14:08:18.258940 32554 raft_consensus.cc:357] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c1a54aa076402181b59b73b4847e60" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 33959 } }
I20250809 14:08:18.259117 32554 raft_consensus.cc:738] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c1a54aa076402181b59b73b4847e60, State: Initialized, Role: FOLLOWER
I20250809 14:08:18.259239 32554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c1a54aa076402181b59b73b4847e60" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 33959 } }
I20250809 14:08:18.259305 32554 raft_consensus.cc:397] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:18.259351 32554 raft_consensus.cc:491] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:18.259393 32554 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 1 FOLLOWER]: Advancing to term 2
I20250809 14:08:18.260130 32554 raft_consensus.cc:513] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c1a54aa076402181b59b73b4847e60" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 33959 } }
I20250809 14:08:18.260226 32554 leader_election.cc:304] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68c1a54aa076402181b59b73b4847e60; no voters:
I20250809 14:08:18.260360 32554 leader_election.cc:290] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [CANDIDATE]: Term 2 election: Requested vote from peers
I20250809 14:08:18.260402 32558 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 2 FOLLOWER]: Leader election won for term 2
I20250809 14:08:18.260506 32558 raft_consensus.cc:695] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [term 2 LEADER]: Becoming Leader. State: Replica: 68c1a54aa076402181b59b73b4847e60, State: Running, Role: LEADER
I20250809 14:08:18.260589 32554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [sys.catalog]: configured and running, proceeding with master startup.
I20250809 14:08:18.260579 32558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c1a54aa076402181b59b73b4847e60" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 33959 } }
I20250809 14:08:18.260843 32559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "68c1a54aa076402181b59b73b4847e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c1a54aa076402181b59b73b4847e60" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 33959 } } }
I20250809 14:08:18.260955 32559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:18.261236 32560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68c1a54aa076402181b59b73b4847e60. Latest consensus state: current_term: 2 leader_uuid: "68c1a54aa076402181b59b73b4847e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68c1a54aa076402181b59b73b4847e60" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 33959 } } }
I20250809 14:08:18.261322 32560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:18.261425 32562 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250809 14:08:18.261927 32562 catalog_manager.cc:671] Loaded metadata for table TestTable [id=360632eed4e84fac85aff7f5c23600de]
I20250809 14:08:18.262101 32562 tablet_loader.cc:96] loaded metadata for tablet 53e7161edc9d4aecb6672d0c7d3484eb (table TestTable [id=360632eed4e84fac85aff7f5c23600de])
I20250809 14:08:18.262159 32562 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250809 14:08:18.262229 32562 catalog_manager.cc:1261] Loaded cluster ID: 81c4a7142151457fb3f0543863312210
I20250809 14:08:18.262257 32562 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250809 14:08:18.263566 32562 catalog_manager.cc:1506] Loading token signing keys...
I20250809 14:08:18.263718 32562 catalog_manager.cc:5966] T 00000000000000000000000000000000 P 68c1a54aa076402181b59b73b4847e60: Loaded TSK: 0
I20250809 14:08:18.263895 32562 catalog_manager.cc:1516] Initializing in-progress tserver states...
W20250809 14:08:18.324519 32556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:18.324687 32556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:18.326076 32556 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20250809 14:08:18.340946 32147 tablet_service.cc:1905] Received UnsafeChangeConfig RPC: dest_uuid: "d8f343363bd4454fa9122b98174782a0"
tablet_id: "53e7161edc9d4aecb6672d0c7d3484eb"
caller_id: "kudu-tools"
new_config {
peers {
permanent_uuid: "178860eeed854ca48128d5588ecc77c4"
}
peers {
permanent_uuid: "d8f343363bd4454fa9122b98174782a0"
}
}
from {username='slave'} at 127.0.0.1:47780
W20250809 14:08:18.341163 32147 raft_consensus.cc:2216] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 1 FOLLOWER]: PROCEEDING WITH UNSAFE CONFIG CHANGE ON THIS SERVER, COMMITTED CONFIG: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } }NEW CONFIG: opid_index: 1 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } unsafe_config_change: true
I20250809 14:08:18.341264 32147 raft_consensus.cc:3058] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 1 FOLLOWER]: Advancing to term 2
I20250809 14:08:18.346964 32518 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "d8f343363bd4454fa9122b98174782a0" instance_seqno: 1754748497374120) as {username='slave'} at 127.31.21.194:43301; Asking this server to re-register.
I20250809 14:08:18.347318 32193 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:18.347394 32193 heartbeater.cc:507] Master 127.31.21.254:33959 requested a full tablet report, sending...
I20250809 14:08:18.347699 32518 ts_manager.cc:194] Registered new tserver with Master: d8f343363bd4454fa9122b98174782a0 (127.31.21.194:36835)
I20250809 14:08:18.495362 32324 heartbeater.cc:499] Master 127.31.21.254:33959 was elected leader, sending a full tablet report...
I20250809 14:08:18.498150 32518 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" instance_seqno: 1754748497487529) as {username='slave'} at 127.31.21.195:39477; Asking this server to re-register.
I20250809 14:08:18.498445 32324 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:18.498521 32324 heartbeater.cc:507] Master 127.31.21.254:33959 requested a full tablet report, sending...
I20250809 14:08:18.498756 32518 ts_manager.cc:194] Registered new tserver with Master: 5e59681166ff4bbdb87acea5800e3ae9 (127.31.21.195:42179)
I20250809 14:08:19.123631 32518 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "178860eeed854ca48128d5588ecc77c4" instance_seqno: 1754748497244357) as {username='slave'} at 127.31.21.193:48371; Asking this server to re-register.
I20250809 14:08:19.123895 32062 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:19.123972 32062 heartbeater.cc:507] Master 127.31.21.254:33959 requested a full tablet report, sending...
I20250809 14:08:19.124209 32518 ts_manager.cc:194] Registered new tserver with Master: 178860eeed854ca48128d5588ecc77c4 (127.31.21.193:46367)
I20250809 14:08:19.621153 32600 raft_consensus.cc:491] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 901a5a1c50f145e69dbc4ea9757d607b)
I20250809 14:08:19.621261 32600 raft_consensus.cc:513] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } }
I20250809 14:08:19.621563 32600 leader_election.cc:290] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 901a5a1c50f145e69dbc4ea9757d607b (127.31.21.196:40077), d8f343363bd4454fa9122b98174782a0 (127.31.21.194:36835)
W20250809 14:08:19.621949 31950 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.21.196:40077: connect: Connection refused (error 111)
W20250809 14:08:19.622330 31950 leader_election.cc:336] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 901a5a1c50f145e69dbc4ea9757d607b (127.31.21.196:40077): Network error: Client connection negotiation failed: client connection to 127.31.21.196:40077: connect: Connection refused (error 111)
I20250809 14:08:19.624164 32147 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "53e7161edc9d4aecb6672d0c7d3484eb" candidate_uuid: "178860eeed854ca48128d5588ecc77c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d8f343363bd4454fa9122b98174782a0" is_pre_election: true
I20250809 14:08:19.624388 31950 leader_election.cc:304] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 178860eeed854ca48128d5588ecc77c4; no voters: 901a5a1c50f145e69dbc4ea9757d607b, d8f343363bd4454fa9122b98174782a0
I20250809 14:08:19.624516 32600 raft_consensus.cc:2747] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250809 14:08:19.842931 32604 raft_consensus.cc:491] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader kudu-tools)
I20250809 14:08:19.843036 32604 raft_consensus.cc:513] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 1 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } unsafe_config_change: true
I20250809 14:08:19.843294 32604 leader_election.cc:290] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 178860eeed854ca48128d5588ecc77c4 (127.31.21.193:46367)
I20250809 14:08:19.845981 32016 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "53e7161edc9d4aecb6672d0c7d3484eb" candidate_uuid: "d8f343363bd4454fa9122b98174782a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "178860eeed854ca48128d5588ecc77c4" is_pre_election: true
I20250809 14:08:19.846081 32016 raft_consensus.cc:2466] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8f343363bd4454fa9122b98174782a0 in term 1.
I20250809 14:08:19.846264 32082 leader_election.cc:304] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 178860eeed854ca48128d5588ecc77c4, d8f343363bd4454fa9122b98174782a0; no voters:
I20250809 14:08:19.846400 32604 raft_consensus.cc:2802] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250809 14:08:19.846454 32604 raft_consensus.cc:491] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 2 FOLLOWER]: Starting leader election (detected failure of leader kudu-tools)
I20250809 14:08:19.846470 32604 raft_consensus.cc:3058] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 2 FOLLOWER]: Advancing to term 3
I20250809 14:08:19.847047 32604 raft_consensus.cc:513] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 1 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } unsafe_config_change: true
I20250809 14:08:19.847151 32604 leader_election.cc:290] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [CANDIDATE]: Term 3 election: Requested vote from peers 178860eeed854ca48128d5588ecc77c4 (127.31.21.193:46367)
I20250809 14:08:19.847319 32016 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "53e7161edc9d4aecb6672d0c7d3484eb" candidate_uuid: "d8f343363bd4454fa9122b98174782a0" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "178860eeed854ca48128d5588ecc77c4"
I20250809 14:08:19.847381 32016 raft_consensus.cc:3058] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 1 FOLLOWER]: Advancing to term 3
I20250809 14:08:19.847878 32016 raft_consensus.cc:2466] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8f343363bd4454fa9122b98174782a0 in term 3.
I20250809 14:08:19.848040 32082 leader_election.cc:304] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 2 voters: 2 yes votes; 0 no votes. yes voters: 178860eeed854ca48128d5588ecc77c4, d8f343363bd4454fa9122b98174782a0; no voters:
I20250809 14:08:19.848119 32604 raft_consensus.cc:2802] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 3 FOLLOWER]: Leader election won for term 3
I20250809 14:08:19.848224 32604 raft_consensus.cc:695] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 3 LEADER]: Becoming Leader. State: Replica: d8f343363bd4454fa9122b98174782a0, State: Running, Role: LEADER
I20250809 14:08:19.848304 32604 consensus_queue.cc:237] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 1 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } unsafe_config_change: true
I20250809 14:08:19.849011 32518 catalog_manager.cc:5582] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 reported cstate change: term changed from 1 to 3, leader changed from 901a5a1c50f145e69dbc4ea9757d607b (127.31.21.196) to d8f343363bd4454fa9122b98174782a0 (127.31.21.194), now has a pending config: VOTER d8f343363bd4454fa9122b98174782a0 (127.31.21.194), VOTER 178860eeed854ca48128d5588ecc77c4 (127.31.21.193). New cstate: current_term: 3 leader_uuid: "d8f343363bd4454fa9122b98174782a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "901a5a1c50f145e69dbc4ea9757d607b" member_type: VOTER last_known_addr { host: "127.31.21.196" port: 40077 } } peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 1 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } unsafe_config_change: true }
I20250809 14:08:20.332762 32016 raft_consensus.cc:1273] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 3 FOLLOWER]: Refusing update from remote peer d8f343363bd4454fa9122b98174782a0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20250809 14:08:20.333107 32604 consensus_queue.cc:1035] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250809 14:08:20.333213 32604 consensus_queue.cc:1230] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [LEADER]: Peer 178860eeed854ca48128d5588ecc77c4 log is divergent from this leader: its last log entry 1.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20250809 14:08:20.334055 32607 raft_consensus.cc:2953] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 3 LEADER]: Committing config change with OpId 2.1: config changed from index -1 to 1, VOTER 901a5a1c50f145e69dbc4ea9757d607b (127.31.21.196) evicted. New config: { opid_index: 1 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } unsafe_config_change: true }
I20250809 14:08:20.335383 32518 catalog_manager.cc:5582] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 reported cstate change: config changed from index -1 to 1, VOTER 901a5a1c50f145e69dbc4ea9757d607b (127.31.21.196) evicted, no longer has a pending config: VOTER d8f343363bd4454fa9122b98174782a0 (127.31.21.194), VOTER 178860eeed854ca48128d5588ecc77c4 (127.31.21.193). New cstate: current_term: 3 leader_uuid: "d8f343363bd4454fa9122b98174782a0" committed_config { opid_index: 1 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } health_report { overall_health: HEALTHY } } unsafe_config_change: true }
W20250809 14:08:20.336136 32518 catalog_manager.cc:5774] Failed to send DeleteTablet RPC for tablet 53e7161edc9d4aecb6672d0c7d3484eb on TS 901a5a1c50f145e69dbc4ea9757d607b: Not found: failed to reset TS proxy: Could not find TS for UUID 901a5a1c50f145e69dbc4ea9757d607b
I20250809 14:08:20.339293 32147 consensus_queue.cc:237] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } peers { permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" member_type: NON_VOTER last_known_addr { host: "127.31.21.195" port: 42179 } attrs { promote: true } } unsafe_config_change: true
I20250809 14:08:20.339759 32016 raft_consensus.cc:1273] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 3 FOLLOWER]: Refusing update from remote peer d8f343363bd4454fa9122b98174782a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20250809 14:08:20.339946 32607 consensus_queue.cc:1035] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250809 14:08:20.340662 32604 raft_consensus.cc:2953] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index 1 to 3, NON_VOTER 5e59681166ff4bbdb87acea5800e3ae9 (127.31.21.195) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } peers { permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" member_type: NON_VOTER last_known_addr { host: "127.31.21.195" port: 42179 } attrs { promote: true } } unsafe_config_change: true }
I20250809 14:08:20.340785 32016 raft_consensus.cc:2953] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index 1 to 3, NON_VOTER 5e59681166ff4bbdb87acea5800e3ae9 (127.31.21.195) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } peers { permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" member_type: NON_VOTER last_known_addr { host: "127.31.21.195" port: 42179 } attrs { promote: true } } unsafe_config_change: true }
I20250809 14:08:20.341993 32518 catalog_manager.cc:5582] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 reported cstate change: config changed from index 1 to 3, NON_VOTER 5e59681166ff4bbdb87acea5800e3ae9 (127.31.21.195) added. New cstate: current_term: 3 leader_uuid: "d8f343363bd4454fa9122b98174782a0" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } peers { permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" member_type: NON_VOTER last_known_addr { host: "127.31.21.195" port: 42179 } attrs { promote: true } } unsafe_config_change: true }
I20250809 14:08:20.342098 32503 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 53e7161edc9d4aecb6672d0c7d3484eb with cas_config_opid_index 1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
W20250809 14:08:20.342974 32081 consensus_peers.cc:489] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 -> Peer 5e59681166ff4bbdb87acea5800e3ae9 (127.31.21.195:42179): Couldn't send request to peer 5e59681166ff4bbdb87acea5800e3ae9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53e7161edc9d4aecb6672d0c7d3484eb. This is attempt 1: this message will repeat every 5th retry.
W20250809 14:08:20.388530 32503 catalog_manager.cc:4726] Async tablet task DeleteTablet RPC for tablet 53e7161edc9d4aecb6672d0c7d3484eb on TS 901a5a1c50f145e69dbc4ea9757d607b failed: Not found: failed to reset TS proxy: Could not find TS for UUID 901a5a1c50f145e69dbc4ea9757d607b
I20250809 14:08:20.793146 32620 ts_tablet_manager.cc:927] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9: Initiating tablet copy from peer d8f343363bd4454fa9122b98174782a0 (127.31.21.194:36835)
I20250809 14:08:20.793375 32620 tablet_copy_client.cc:323] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9: tablet copy: Beginning tablet copy session from remote peer at address 127.31.21.194:36835
I20250809 14:08:20.795776 32167 tablet_copy_service.cc:140] P d8f343363bd4454fa9122b98174782a0: Received BeginTabletCopySession request for tablet 53e7161edc9d4aecb6672d0c7d3484eb from peer 5e59681166ff4bbdb87acea5800e3ae9 ({username='slave'} at 127.31.21.195:40059)
I20250809 14:08:20.795841 32167 tablet_copy_service.cc:161] P d8f343363bd4454fa9122b98174782a0: Beginning new tablet copy session on tablet 53e7161edc9d4aecb6672d0c7d3484eb from peer 5e59681166ff4bbdb87acea5800e3ae9 at {username='slave'} at 127.31.21.195:40059: session id = 5e59681166ff4bbdb87acea5800e3ae9-53e7161edc9d4aecb6672d0c7d3484eb
I20250809 14:08:20.796370 32167 tablet_copy_source_session.cc:215] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0: Tablet Copy: opened 0 blocks and 1 log segments
I20250809 14:08:20.797086 32620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e7161edc9d4aecb6672d0c7d3484eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:20.798898 32620 tablet_copy_client.cc:806] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9: tablet copy: Starting download of 0 data blocks...
I20250809 14:08:20.799039 32620 tablet_copy_client.cc:670] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9: tablet copy: Starting download of 1 WAL segments...
I20250809 14:08:20.799670 32620 tablet_copy_client.cc:538] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250809 14:08:20.800247 32620 tablet_bootstrap.cc:492] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9: Bootstrap starting.
I20250809 14:08:20.801250 32620 log.cc:826] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:20.802129 32620 tablet_bootstrap.cc:492] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:20.802299 32620 tablet_bootstrap.cc:492] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9: Bootstrap complete.
I20250809 14:08:20.802379 32620 ts_tablet_manager.cc:1397] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:20.803663 32620 raft_consensus.cc:357] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } peers { permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" member_type: NON_VOTER last_known_addr { host: "127.31.21.195" port: 42179 } attrs { promote: true } } unsafe_config_change: true
I20250809 14:08:20.803783 32620 raft_consensus.cc:738] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 5e59681166ff4bbdb87acea5800e3ae9, State: Initialized, Role: LEARNER
I20250809 14:08:20.803826 32620 consensus_queue.cc:260] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } peers { permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" member_type: NON_VOTER last_known_addr { host: "127.31.21.195" port: 42179 } attrs { promote: true } } unsafe_config_change: true
I20250809 14:08:20.804008 32620 ts_tablet_manager.cc:1428] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250809 14:08:20.804240 32167 tablet_copy_service.cc:342] P d8f343363bd4454fa9122b98174782a0: Request end of tablet copy session 5e59681166ff4bbdb87acea5800e3ae9-53e7161edc9d4aecb6672d0c7d3484eb received from {username='slave'} at 127.31.21.195:40059
I20250809 14:08:20.804296 32167 tablet_copy_service.cc:434] P d8f343363bd4454fa9122b98174782a0: ending tablet copy session 5e59681166ff4bbdb87acea5800e3ae9-53e7161edc9d4aecb6672d0c7d3484eb on tablet 53e7161edc9d4aecb6672d0c7d3484eb with peer 5e59681166ff4bbdb87acea5800e3ae9
I20250809 14:08:21.195099 32278 raft_consensus.cc:1215] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3] Dedup: 3.3->[]
W20250809 14:08:21.554404 32503 catalog_manager.cc:4726] Async tablet task DeleteTablet RPC for tablet 53e7161edc9d4aecb6672d0c7d3484eb on TS 901a5a1c50f145e69dbc4ea9757d607b failed: Not found: failed to reset TS proxy: Could not find TS for UUID 901a5a1c50f145e69dbc4ea9757d607b
I20250809 14:08:21.728266 32607 raft_consensus.cc:1062] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0: attempting to promote NON_VOTER 5e59681166ff4bbdb87acea5800e3ae9 to VOTER
I20250809 14:08:21.728488 32607 consensus_queue.cc:237] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } peers { permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 42179 } attrs { promote: false } } unsafe_config_change: true
I20250809 14:08:21.728986 32016 raft_consensus.cc:1273] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 3 FOLLOWER]: Refusing update from remote peer d8f343363bd4454fa9122b98174782a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20250809 14:08:21.729005 32278 raft_consensus.cc:1273] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9 [term 3 LEARNER]: Refusing update from remote peer d8f343363bd4454fa9122b98174782a0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20250809 14:08:21.729210 32628 consensus_queue.cc:1035] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 42179 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250809 14:08:21.729370 32607 consensus_queue.cc:1035] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20250809 14:08:21.730106 32607 raft_consensus.cc:2953] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 5e59681166ff4bbdb87acea5800e3ae9 (127.31.21.195) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } peers { permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 42179 } attrs { promote: false } } unsafe_config_change: true }
I20250809 14:08:21.730245 32016 raft_consensus.cc:2953] T 53e7161edc9d4aecb6672d0c7d3484eb P 178860eeed854ca48128d5588ecc77c4 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 5e59681166ff4bbdb87acea5800e3ae9 (127.31.21.195) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } peers { permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 42179 } attrs { promote: false } } unsafe_config_change: true }
I20250809 14:08:21.730456 32278 raft_consensus.cc:2953] T 53e7161edc9d4aecb6672d0c7d3484eb P 5e59681166ff4bbdb87acea5800e3ae9 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 5e59681166ff4bbdb87acea5800e3ae9 (127.31.21.195) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } } peers { permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 42179 } attrs { promote: false } } unsafe_config_change: true }
I20250809 14:08:21.731379 32517 catalog_manager.cc:5582] T 53e7161edc9d4aecb6672d0c7d3484eb P d8f343363bd4454fa9122b98174782a0 reported cstate change: config changed from index 3 to 4, 5e59681166ff4bbdb87acea5800e3ae9 (127.31.21.195) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "d8f343363bd4454fa9122b98174782a0" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d8f343363bd4454fa9122b98174782a0" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 36835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "178860eeed854ca48128d5588ecc77c4" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 46367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 42179 } attrs { promote: false } health_report { overall_health: HEALTHY } } unsafe_config_change: true }
I20250809 14:08:21.750290 31831 kudu-admin-test.cc:751] Waiting for Master to see new config...
I20250809 14:08:21.753662 31831 kudu-admin-test.cc:756] Tablet locations:
tablet_locations {
tablet_id: "53e7161edc9d4aecb6672d0c7d3484eb"
DEPRECATED_stale: false
partition {
partition_key_start: ""
partition_key_end: ""
}
interned_replicas {
ts_info_idx: 0
role: LEADER
}
interned_replicas {
ts_info_idx: 1
role: FOLLOWER
}
interned_replicas {
ts_info_idx: 2
role: FOLLOWER
}
}
ts_infos {
permanent_uuid: "d8f343363bd4454fa9122b98174782a0"
rpc_addresses {
host: "127.31.21.194"
port: 36835
}
}
ts_infos {
permanent_uuid: "178860eeed854ca48128d5588ecc77c4"
rpc_addresses {
host: "127.31.21.193"
port: 46367
}
}
ts_infos {
permanent_uuid: "5e59681166ff4bbdb87acea5800e3ae9"
rpc_addresses {
host: "127.31.21.195"
port: 42179
}
}
I20250809 14:08:21.754081 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 31934
I20250809 14:08:21.759166 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 32065
I20250809 14:08:21.763928 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 32196
I20250809 14:08:21.770112 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 32487
2025-08-09T14:08:21Z chronyd exiting
[ OK ] AdminCliTest.TestUnsafeChangeConfigForConfigWithTwoNodes (4793 ms)
[ RUN ] AdminCliTest.TestGracefulSpecificLeaderStepDown
I20250809 14:08:21.790784 31831 test_util.cc:276] Using random seed: 958304315
I20250809 14:08:21.791260 31831 ts_itest-base.cc:115] Starting cluster with:
I20250809 14:08:21.791301 31831 ts_itest-base.cc:116] --------------
I20250809 14:08:21.791313 31831 ts_itest-base.cc:117] 3 tablet servers
I20250809 14:08:21.791323 31831 ts_itest-base.cc:118] 3 replicas per TS
I20250809 14:08:21.791339 31831 ts_itest-base.cc:119] --------------
2025-08-09T14:08:21Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-09T14:08:21Z Disabled control of system clock
I20250809 14:08:21.798061 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:37437
--webserver_interface=127.31.21.254
--webserver_port=0
--builtin_ntp_servers=127.31.21.212:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.21.254:37437
--catalog_manager_wait_for_new_tablets_to_elect_leader=false with env {}
W20250809 14:08:21.873185 32646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:21.873347 32646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:21.873378 32646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:21.874687 32646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250809 14:08:21.874727 32646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:21.874747 32646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250809 14:08:21.874763 32646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250809 14:08:21.876127 32646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.31.21.254:37437
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:37437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.21.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.32646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:21.876343 32646 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:21.876538 32646 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:21.878820 32654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:21.878849 32651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:21.878890 32652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:21.878985 32646 server_base.cc:1047] running on GCE node
I20250809 14:08:21.879289 32646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:21.879496 32646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:21.880620 32646 hybrid_clock.cc:648] HybridClock initialized: now 1754748501880605 us; error 31 us; skew 500 ppm
I20250809 14:08:21.881716 32646 webserver.cc:489] Webserver started at http://127.31.21.254:40369/ using document root <none> and password file <none>
I20250809 14:08:21.881894 32646 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:21.881935 32646 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:21.882048 32646 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:21.882880 32646 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "67dc4478fd29420598e78af20d240c3a"
format_stamp: "Formatted at 2025-08-09 14:08:21 on dist-test-slave-mstq"
I20250809 14:08:21.883157 32646 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "67dc4478fd29420598e78af20d240c3a"
format_stamp: "Formatted at 2025-08-09 14:08:21 on dist-test-slave-mstq"
I20250809 14:08:21.884239 32646 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250809 14:08:21.884832 32660 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:21.885006 32646 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:21.885068 32646 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
uuid: "67dc4478fd29420598e78af20d240c3a"
format_stamp: "Formatted at 2025-08-09 14:08:21 on dist-test-slave-mstq"
I20250809 14:08:21.885114 32646 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:21.894367 32646 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:21.894578 32646 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:21.894658 32646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:21.898205 32646 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.254:37437
I20250809 14:08:21.898236 32712 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.254:37437 every 8 connection(s)
I20250809 14:08:21.898523 32646 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250809 14:08:21.899053 32713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:21.901274 32713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a: Bootstrap starting.
I20250809 14:08:21.901690 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 32646
I20250809 14:08:21.901779 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250809 14:08:21.901888 32713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:21.902171 32713 log.cc:826] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:21.902820 32713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a: No bootstrap required, opened a new log
I20250809 14:08:21.904009 32713 raft_consensus.cc:357] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67dc4478fd29420598e78af20d240c3a" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37437 } }
I20250809 14:08:21.904124 32713 raft_consensus.cc:383] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:21.904155 32713 raft_consensus.cc:738] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67dc4478fd29420598e78af20d240c3a, State: Initialized, Role: FOLLOWER
I20250809 14:08:21.904243 32713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67dc4478fd29420598e78af20d240c3a" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37437 } }
I20250809 14:08:21.904305 32713 raft_consensus.cc:397] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:21.904361 32713 raft_consensus.cc:491] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:21.904412 32713 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:21.904902 32713 raft_consensus.cc:513] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67dc4478fd29420598e78af20d240c3a" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37437 } }
I20250809 14:08:21.905001 32713 leader_election.cc:304] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67dc4478fd29420598e78af20d240c3a; no voters:
I20250809 14:08:21.905133 32713 leader_election.cc:290] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:21.905181 32718 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:21.905409 32713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [sys.catalog]: configured and running, proceeding with master startup.
I20250809 14:08:21.905557 32718 raft_consensus.cc:695] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [term 1 LEADER]: Becoming Leader. State: Replica: 67dc4478fd29420598e78af20d240c3a, State: Running, Role: LEADER
I20250809 14:08:21.905676 32718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67dc4478fd29420598e78af20d240c3a" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37437 } }
I20250809 14:08:21.905968 32719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67dc4478fd29420598e78af20d240c3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67dc4478fd29420598e78af20d240c3a" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37437 } } }
I20250809 14:08:21.906136 32719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:21.906157 32720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67dc4478fd29420598e78af20d240c3a. Latest consensus state: current_term: 1 leader_uuid: "67dc4478fd29420598e78af20d240c3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67dc4478fd29420598e78af20d240c3a" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37437 } } }
I20250809 14:08:21.906291 32720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:21.906682 32727 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250809 14:08:21.907246 32727 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250809 14:08:21.908450 32727 catalog_manager.cc:1349] Generated new cluster ID: 6fb376daafe14ea88055b77e05ff217b
I20250809 14:08:21.908504 32727 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250809 14:08:21.930573 32727 catalog_manager.cc:1372] Generated new certificate authority record
I20250809 14:08:21.931114 32727 catalog_manager.cc:1506] Loading token signing keys...
I20250809 14:08:21.934410 32727 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 67dc4478fd29420598e78af20d240c3a: Generated new TSK 0
I20250809 14:08:21.934598 32727 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250809 14:08:21.939309 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.193:0
--local_ip_for_outbound_sockets=127.31.21.193
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:37437
--builtin_ntp_servers=127.31.21.212:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20250809 14:08:22.025552 32737 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20250809 14:08:22.025699 32737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:22.025722 32737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:22.025748 32737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:22.026949 32737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:22.026994 32737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.193
I20250809 14:08:22.028240 32737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:37437
--never_fsync=true
--heap_profile_path=/tmp/kudu.32737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.193
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:22.028437 32737 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:22.028623 32737 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:22.030951 32742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:22.030941 32745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:22.031069 32737 server_base.cc:1047] running on GCE node
W20250809 14:08:22.030941 32743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:22.031296 32737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:22.031503 32737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:22.032651 32737 hybrid_clock.cc:648] HybridClock initialized: now 1754748502032633 us; error 30 us; skew 500 ppm
I20250809 14:08:22.034025 32737 webserver.cc:489] Webserver started at http://127.31.21.193:38399/ using document root <none> and password file <none>
I20250809 14:08:22.034235 32737 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:22.034279 32737 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:22.034381 32737 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:22.035105 32737 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "e904215dc8f542f085b87094d3ea9bc3"
format_stamp: "Formatted at 2025-08-09 14:08:22 on dist-test-slave-mstq"
I20250809 14:08:22.035328 32737 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "e904215dc8f542f085b87094d3ea9bc3"
format_stamp: "Formatted at 2025-08-09 14:08:22 on dist-test-slave-mstq"
I20250809 14:08:22.036312 32737 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250809 14:08:22.036933 32751 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:22.037091 32737 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250809 14:08:22.037186 32737 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "e904215dc8f542f085b87094d3ea9bc3"
format_stamp: "Formatted at 2025-08-09 14:08:22 on dist-test-slave-mstq"
I20250809 14:08:22.037241 32737 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:22.047513 32737 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:22.047691 32737 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:22.047788 32737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:22.047969 32737 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:22.048220 32737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:22.048249 32737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:22.048277 32737 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:22.048302 32737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:22.053344 32737 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.193:39323
I20250809 14:08:22.053390 396 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.193:39323 every 8 connection(s)
I20250809 14:08:22.053658 32737 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250809 14:08:22.058125 397 heartbeater.cc:344] Connected to a master server at 127.31.21.254:37437
I20250809 14:08:22.058216 397 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:22.058394 397 heartbeater.cc:507] Master 127.31.21.254:37437 requested a full tablet report, sending...
I20250809 14:08:22.058810 32677 ts_manager.cc:194] Registered new tserver with Master: e904215dc8f542f085b87094d3ea9bc3 (127.31.21.193:39323)
I20250809 14:08:22.059767 32677 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.193:44489
I20250809 14:08:22.063791 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 32737
I20250809 14:08:22.063889 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250809 14:08:22.064903 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.194:0
--local_ip_for_outbound_sockets=127.31.21.194
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:37437
--builtin_ntp_servers=127.31.21.212:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20250809 14:08:22.138693 400 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20250809 14:08:22.138831 400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:22.138845 400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:22.138880 400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:22.140230 400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:22.140275 400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.194
I20250809 14:08:22.141796 400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:37437
--never_fsync=true
--heap_profile_path=/tmp/kudu.400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.194
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:22.142022 400 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:22.142268 400 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:22.144989 406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:22.145011 408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:22.145079 405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:22.145354 400 server_base.cc:1047] running on GCE node
I20250809 14:08:22.145493 400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:22.145684 400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:22.146809 400 hybrid_clock.cc:648] HybridClock initialized: now 1754748502146796 us; error 26 us; skew 500 ppm
I20250809 14:08:22.147835 400 webserver.cc:489] Webserver started at http://127.31.21.194:43353/ using document root <none> and password file <none>
I20250809 14:08:22.148015 400 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:22.148056 400 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:22.148176 400 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:22.149031 400 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "26d19a86cf894d52ba508f893017df6f"
format_stamp: "Formatted at 2025-08-09 14:08:22 on dist-test-slave-mstq"
I20250809 14:08:22.149327 400 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "26d19a86cf894d52ba508f893017df6f"
format_stamp: "Formatted at 2025-08-09 14:08:22 on dist-test-slave-mstq"
I20250809 14:08:22.150419 400 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:22.151026 414 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:22.151176 400 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:22.151244 400 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "26d19a86cf894d52ba508f893017df6f"
format_stamp: "Formatted at 2025-08-09 14:08:22 on dist-test-slave-mstq"
I20250809 14:08:22.151297 400 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:22.174958 400 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:22.175246 400 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:22.175362 400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:22.175586 400 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:22.175930 400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:22.175969 400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:22.176016 400 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:22.176043 400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:22.181721 400 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.194:41939
I20250809 14:08:22.181773 527 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.194:41939 every 8 connection(s)
I20250809 14:08:22.182068 400 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250809 14:08:22.186160 528 heartbeater.cc:344] Connected to a master server at 127.31.21.254:37437
I20250809 14:08:22.186251 528 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:22.186443 528 heartbeater.cc:507] Master 127.31.21.254:37437 requested a full tablet report, sending...
I20250809 14:08:22.186738 32677 ts_manager.cc:194] Registered new tserver with Master: 26d19a86cf894d52ba508f893017df6f (127.31.21.194:41939)
I20250809 14:08:22.187072 32677 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.194:58641
I20250809 14:08:22.189343 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 400
I20250809 14:08:22.189411 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250809 14:08:22.190315 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.195:0
--local_ip_for_outbound_sockets=127.31.21.195
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:37437
--builtin_ntp_servers=127.31.21.212:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20250809 14:08:22.269261 531 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20250809 14:08:22.269411 531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:22.269425 531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:22.269440 531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:22.270673 531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:22.270712 531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.195
I20250809 14:08:22.272034 531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:37437
--never_fsync=true
--heap_profile_path=/tmp/kudu.531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.195
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:22.272183 531 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:22.272360 531 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:22.274700 539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:22.274729 536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:22.274731 537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:22.274956 531 server_base.cc:1047] running on GCE node
I20250809 14:08:22.275072 531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:22.275218 531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:22.276326 531 hybrid_clock.cc:648] HybridClock initialized: now 1754748502276296 us; error 46 us; skew 500 ppm
I20250809 14:08:22.277367 531 webserver.cc:489] Webserver started at http://127.31.21.195:43361/ using document root <none> and password file <none>
I20250809 14:08:22.277542 531 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:22.277582 531 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:22.277688 531 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:22.278504 531 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "396f9d1e89814956b1fc094f15b9a584"
format_stamp: "Formatted at 2025-08-09 14:08:22 on dist-test-slave-mstq"
I20250809 14:08:22.278795 531 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "396f9d1e89814956b1fc094f15b9a584"
format_stamp: "Formatted at 2025-08-09 14:08:22 on dist-test-slave-mstq"
I20250809 14:08:22.279812 531 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20250809 14:08:22.280381 545 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:22.280517 531 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:22.280583 531 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "396f9d1e89814956b1fc094f15b9a584"
format_stamp: "Formatted at 2025-08-09 14:08:22 on dist-test-slave-mstq"
I20250809 14:08:22.280632 531 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:22.312629 531 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:22.312902 531 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:22.313015 531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:22.313238 531 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:22.313537 531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:22.313570 531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:22.313604 531 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:22.313630 531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:22.318821 531 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.195:45645
I20250809 14:08:22.318881 658 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.195:45645 every 8 connection(s)
I20250809 14:08:22.319150 531 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250809 14:08:22.323086 659 heartbeater.cc:344] Connected to a master server at 127.31.21.254:37437
I20250809 14:08:22.323171 659 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:22.323354 659 heartbeater.cc:507] Master 127.31.21.254:37437 requested a full tablet report, sending...
I20250809 14:08:22.323652 32677 ts_manager.cc:194] Registered new tserver with Master: 396f9d1e89814956b1fc094f15b9a584 (127.31.21.195:45645)
I20250809 14:08:22.323989 32677 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.195:36155
I20250809 14:08:22.324739 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 531
I20250809 14:08:22.324806 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestGracefulSpecificLeaderStepDown.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250809 14:08:22.325609 31831 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250809 14:08:22.330483 32677 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46414:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
W20250809 14:08:22.331617 32677 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250809 14:08:22.337555 593 tablet_service.cc:1468] Processing CreateTablet for tablet 64e43017a634455586a2d691513df314 (DEFAULT_TABLE table=TestTable [id=f634bbb7775c4ac8bbc7c7d4f651071e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:22.337817 462 tablet_service.cc:1468] Processing CreateTablet for tablet 64e43017a634455586a2d691513df314 (DEFAULT_TABLE table=TestTable [id=f634bbb7775c4ac8bbc7c7d4f651071e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:22.337886 593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e43017a634455586a2d691513df314. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:22.338076 462 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e43017a634455586a2d691513df314. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:22.338204 331 tablet_service.cc:1468] Processing CreateTablet for tablet 64e43017a634455586a2d691513df314 (DEFAULT_TABLE table=TestTable [id=f634bbb7775c4ac8bbc7c7d4f651071e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:22.338476 331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64e43017a634455586a2d691513df314. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:22.339794 673 tablet_bootstrap.cc:492] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584: Bootstrap starting.
I20250809 14:08:22.340338 673 tablet_bootstrap.cc:654] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:22.340572 673 log.cc:826] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:22.340619 674 tablet_bootstrap.cc:492] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f: Bootstrap starting.
I20250809 14:08:22.341202 676 tablet_bootstrap.cc:492] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3: Bootstrap starting.
I20250809 14:08:22.341254 674 tablet_bootstrap.cc:654] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:22.341315 673 tablet_bootstrap.cc:492] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584: No bootstrap required, opened a new log
I20250809 14:08:22.341382 673 ts_tablet_manager.cc:1397] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:22.341526 674 log.cc:826] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:22.341892 676 tablet_bootstrap.cc:654] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:22.342031 674 tablet_bootstrap.cc:492] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f: No bootstrap required, opened a new log
I20250809 14:08:22.342070 674 ts_tablet_manager.cc:1397] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250809 14:08:22.342157 676 log.cc:826] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:22.342808 676 tablet_bootstrap.cc:492] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3: No bootstrap required, opened a new log
I20250809 14:08:22.342857 676 ts_tablet_manager.cc:1397] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:22.342991 673 raft_consensus.cc:357] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } }
I20250809 14:08:22.343111 673 raft_consensus.cc:738] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396f9d1e89814956b1fc094f15b9a584, State: Initialized, Role: FOLLOWER
I20250809 14:08:22.343132 674 raft_consensus.cc:357] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } }
I20250809 14:08:22.343163 673 consensus_queue.cc:260] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } }
I20250809 14:08:22.343223 674 raft_consensus.cc:738] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26d19a86cf894d52ba508f893017df6f, State: Initialized, Role: FOLLOWER
I20250809 14:08:22.343257 674 consensus_queue.cc:260] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } }
I20250809 14:08:22.343379 673 ts_tablet_manager.cc:1428] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:22.343425 659 heartbeater.cc:499] Master 127.31.21.254:37437 was elected leader, sending a full tablet report...
I20250809 14:08:22.343443 674 ts_tablet_manager.cc:1428] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20250809 14:08:22.343528 528 heartbeater.cc:499] Master 127.31.21.254:37437 was elected leader, sending a full tablet report...
I20250809 14:08:22.344388 676 raft_consensus.cc:357] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } }
I20250809 14:08:22.344506 676 raft_consensus.cc:738] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e904215dc8f542f085b87094d3ea9bc3, State: Initialized, Role: FOLLOWER
I20250809 14:08:22.344560 676 consensus_queue.cc:260] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } }
I20250809 14:08:22.344781 676 ts_tablet_manager.cc:1428] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:22.344907 397 heartbeater.cc:499] Master 127.31.21.254:37437 was elected leader, sending a full tablet report...
I20250809 14:08:22.348548 31831 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250809 14:08:22.348925 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver e904215dc8f542f085b87094d3ea9bc3 to finish bootstrapping
I20250809 14:08:22.352142 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 26d19a86cf894d52ba508f893017df6f to finish bootstrapping
I20250809 14:08:22.355094 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 396f9d1e89814956b1fc094f15b9a584 to finish bootstrapping
I20250809 14:08:22.364876 613 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "64e43017a634455586a2d691513df314"
dest_uuid: "396f9d1e89814956b1fc094f15b9a584"
from {username='slave'} at 127.0.0.1:36880
I20250809 14:08:22.364982 613 raft_consensus.cc:491] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20250809 14:08:22.365037 613 raft_consensus.cc:3058] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:22.365645 613 raft_consensus.cc:513] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } }
I20250809 14:08:22.365890 613 leader_election.cc:290] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [CANDIDATE]: Term 1 election: Requested vote from peers e904215dc8f542f085b87094d3ea9bc3 (127.31.21.193:39323), 26d19a86cf894d52ba508f893017df6f (127.31.21.194:41939)
I20250809 14:08:22.367193 31831 cluster_itest_util.cc:257] Not converged past 1 yet: 0.0 0.0 0.0
I20250809 14:08:22.369055 351 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "64e43017a634455586a2d691513df314" candidate_uuid: "396f9d1e89814956b1fc094f15b9a584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e904215dc8f542f085b87094d3ea9bc3"
I20250809 14:08:22.369148 351 raft_consensus.cc:3058] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:22.369127 482 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "64e43017a634455586a2d691513df314" candidate_uuid: "396f9d1e89814956b1fc094f15b9a584" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "26d19a86cf894d52ba508f893017df6f"
I20250809 14:08:22.369212 482 raft_consensus.cc:3058] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:22.370005 351 raft_consensus.cc:2466] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396f9d1e89814956b1fc094f15b9a584 in term 1.
I20250809 14:08:22.370057 482 raft_consensus.cc:2466] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396f9d1e89814956b1fc094f15b9a584 in term 1.
I20250809 14:08:22.370257 547 leader_election.cc:304] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396f9d1e89814956b1fc094f15b9a584, e904215dc8f542f085b87094d3ea9bc3; no voters:
I20250809 14:08:22.370414 679 raft_consensus.cc:2802] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:22.370539 679 raft_consensus.cc:695] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 1 LEADER]: Becoming Leader. State: Replica: 396f9d1e89814956b1fc094f15b9a584, State: Running, Role: LEADER
I20250809 14:08:22.370630 679 consensus_queue.cc:237] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } }
I20250809 14:08:22.371357 32676 catalog_manager.cc:5582] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 reported cstate change: term changed from 0 to 1, leader changed from <none> to 396f9d1e89814956b1fc094f15b9a584 (127.31.21.195). New cstate: current_term: 1 leader_uuid: "396f9d1e89814956b1fc094f15b9a584" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } health_report { overall_health: UNKNOWN } } }
W20250809 14:08:22.432552 529 tablet.cc:2378] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250809 14:08:22.468679 31831 cluster_itest_util.cc:257] Not converged past 1 yet: 1.1 0.0 0.0
W20250809 14:08:22.554302 398 tablet.cc:2378] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250809 14:08:22.569645 660 tablet.cc:2378] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250809 14:08:22.669972 31831 cluster_itest_util.cc:257] Not converged past 1 yet: 1.1 0.0 0.0
I20250809 14:08:22.784684 679 consensus_queue.cc:1035] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250809 14:08:22.788560 689 consensus_queue.cc:1035] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250809 14:08:23.068508 613 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "64e43017a634455586a2d691513df314"
dest_uuid: "396f9d1e89814956b1fc094f15b9a584"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:36890
I20250809 14:08:23.068614 613 raft_consensus.cc:604] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 1 LEADER]: Received request to transfer leadership
I20250809 14:08:23.219295 679 raft_consensus.cc:991] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584: : Instructing follower e904215dc8f542f085b87094d3ea9bc3 to start an election
I20250809 14:08:23.219390 679 raft_consensus.cc:1079] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 1 LEADER]: Signalling peer e904215dc8f542f085b87094d3ea9bc3 to start an election
I20250809 14:08:23.219624 351 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "64e43017a634455586a2d691513df314"
dest_uuid: "e904215dc8f542f085b87094d3ea9bc3"
from {username='slave'} at 127.31.21.195:50809
I20250809 14:08:23.219733 351 raft_consensus.cc:491] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250809 14:08:23.219755 351 raft_consensus.cc:3058] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [term 1 FOLLOWER]: Advancing to term 2
I20250809 14:08:23.220392 351 raft_consensus.cc:513] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } }
I20250809 14:08:23.220665 351 leader_election.cc:290] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [CANDIDATE]: Term 2 election: Requested vote from peers 396f9d1e89814956b1fc094f15b9a584 (127.31.21.195:45645), 26d19a86cf894d52ba508f893017df6f (127.31.21.194:41939)
I20250809 14:08:23.223158 613 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "64e43017a634455586a2d691513df314" candidate_uuid: "e904215dc8f542f085b87094d3ea9bc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "396f9d1e89814956b1fc094f15b9a584"
I20250809 14:08:23.223212 482 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "64e43017a634455586a2d691513df314" candidate_uuid: "e904215dc8f542f085b87094d3ea9bc3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "26d19a86cf894d52ba508f893017df6f"
I20250809 14:08:23.223245 613 raft_consensus.cc:3053] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 1 LEADER]: Stepping down as leader of term 1
I20250809 14:08:23.223274 613 raft_consensus.cc:738] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 396f9d1e89814956b1fc094f15b9a584, State: Running, Role: LEADER
I20250809 14:08:23.223277 482 raft_consensus.cc:3058] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f [term 1 FOLLOWER]: Advancing to term 2
I20250809 14:08:23.223325 613 consensus_queue.cc:260] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } }
I20250809 14:08:23.223420 613 raft_consensus.cc:3058] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 1 FOLLOWER]: Advancing to term 2
I20250809 14:08:23.223848 482 raft_consensus.cc:2466] T 64e43017a634455586a2d691513df314 P 26d19a86cf894d52ba508f893017df6f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e904215dc8f542f085b87094d3ea9bc3 in term 2.
I20250809 14:08:23.223982 613 raft_consensus.cc:2466] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e904215dc8f542f085b87094d3ea9bc3 in term 2.
I20250809 14:08:23.224051 32752 leader_election.cc:304] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26d19a86cf894d52ba508f893017df6f, e904215dc8f542f085b87094d3ea9bc3; no voters:
I20250809 14:08:23.224172 681 raft_consensus.cc:2802] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [term 2 FOLLOWER]: Leader election won for term 2
I20250809 14:08:23.224262 681 raft_consensus.cc:695] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [term 2 LEADER]: Becoming Leader. State: Replica: e904215dc8f542f085b87094d3ea9bc3, State: Running, Role: LEADER
I20250809 14:08:23.224361 681 consensus_queue.cc:237] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } }
I20250809 14:08:23.224902 32677 catalog_manager.cc:5582] T 64e43017a634455586a2d691513df314 P e904215dc8f542f085b87094d3ea9bc3 reported cstate change: term changed from 1 to 2, leader changed from 396f9d1e89814956b1fc094f15b9a584 (127.31.21.195) to e904215dc8f542f085b87094d3ea9bc3 (127.31.21.193). New cstate: current_term: 2 leader_uuid: "e904215dc8f542f085b87094d3ea9bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e904215dc8f542f085b87094d3ea9bc3" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396f9d1e89814956b1fc094f15b9a584" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26d19a86cf894d52ba508f893017df6f" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 41939 } health_report { overall_health: UNKNOWN } } }
I20250809 14:08:23.368021 613 tablet_service.cc:1968] Received LeaderStepDown RPC: tablet_id: "64e43017a634455586a2d691513df314"
dest_uuid: "396f9d1e89814956b1fc094f15b9a584"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:36902
I20250809 14:08:23.368106 613 raft_consensus.cc:604] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 2 FOLLOWER]: Received request to transfer leadership
I20250809 14:08:23.368136 613 raft_consensus.cc:612] T 64e43017a634455586a2d691513df314 P 396f9d1e89814956b1fc094f15b9a584 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20250809 14:08:23.373788 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 32737
I20250809 14:08:23.380067 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 400
I20250809 14:08:23.384503 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 531
I20250809 14:08:23.388728 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 32646
2025-08-09T14:08:23Z chronyd exiting
[ OK ] AdminCliTest.TestGracefulSpecificLeaderStepDown (1617 ms)
[ RUN ] AdminCliTest.TestDescribeTableColumnFlags
I20250809 14:08:23.408082 31831 test_util.cc:276] Using random seed: 959921614
I20250809 14:08:23.408552 31831 ts_itest-base.cc:115] Starting cluster with:
I20250809 14:08:23.408591 31831 ts_itest-base.cc:116] --------------
I20250809 14:08:23.408602 31831 ts_itest-base.cc:117] 3 tablet servers
I20250809 14:08:23.408613 31831 ts_itest-base.cc:118] 3 replicas per TS
I20250809 14:08:23.408630 31831 ts_itest-base.cc:119] --------------
2025-08-09T14:08:23Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-09T14:08:23Z Disabled control of system clock
I20250809 14:08:23.415628 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:38861
--webserver_interface=127.31.21.254
--webserver_port=0
--builtin_ntp_servers=127.31.21.212:36131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.21.254:38861 with env {}
W20250809 14:08:23.489321 738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:23.489470 738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:23.489497 738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:23.490757 738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250809 14:08:23.490797 738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:23.490828 738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250809 14:08:23.490847 738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250809 14:08:23.492282 738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:36131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.21.254:38861
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:38861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.21.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:23.492496 738 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:23.492694 738 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:23.494877 746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:23.494918 744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:23.494935 743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:23.495049 738 server_base.cc:1047] running on GCE node
I20250809 14:08:23.495276 738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:23.495477 738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:23.496606 738 hybrid_clock.cc:648] HybridClock initialized: now 1754748503496587 us; error 33 us; skew 500 ppm
I20250809 14:08:23.497741 738 webserver.cc:489] Webserver started at http://127.31.21.254:42563/ using document root <none> and password file <none>
I20250809 14:08:23.497932 738 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:23.497975 738 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:23.498095 738 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:23.498881 738 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "7c59f412dcbb4a40a67c364f57e57027"
format_stamp: "Formatted at 2025-08-09 14:08:23 on dist-test-slave-mstq"
I20250809 14:08:23.499163 738 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "7c59f412dcbb4a40a67c364f57e57027"
format_stamp: "Formatted at 2025-08-09 14:08:23 on dist-test-slave-mstq"
I20250809 14:08:23.500209 738 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250809 14:08:23.500795 752 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:23.500963 738 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:23.501030 738 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
uuid: "7c59f412dcbb4a40a67c364f57e57027"
format_stamp: "Formatted at 2025-08-09 14:08:23 on dist-test-slave-mstq"
I20250809 14:08:23.501083 738 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:23.513841 738 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:23.514079 738 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:23.514184 738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:23.517597 738 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.254:38861
I20250809 14:08:23.517644 804 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.254:38861 every 8 connection(s)
I20250809 14:08:23.517982 738 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250809 14:08:23.518503 805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:23.519371 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 738
I20250809 14:08:23.519465 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250809 14:08:23.520877 805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027: Bootstrap starting.
I20250809 14:08:23.521409 805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:23.521632 805 log.cc:826] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:23.522182 805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027: No bootstrap required, opened a new log
I20250809 14:08:23.523116 805 raft_consensus.cc:357] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c59f412dcbb4a40a67c364f57e57027" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 38861 } }
I20250809 14:08:23.523264 805 raft_consensus.cc:383] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:23.523296 805 raft_consensus.cc:738] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c59f412dcbb4a40a67c364f57e57027, State: Initialized, Role: FOLLOWER
I20250809 14:08:23.523418 805 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c59f412dcbb4a40a67c364f57e57027" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 38861 } }
I20250809 14:08:23.523516 805 raft_consensus.cc:397] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:23.523569 805 raft_consensus.cc:491] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:23.523622 805 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:23.524283 805 raft_consensus.cc:513] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c59f412dcbb4a40a67c364f57e57027" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 38861 } }
I20250809 14:08:23.524377 805 leader_election.cc:304] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c59f412dcbb4a40a67c364f57e57027; no voters:
I20250809 14:08:23.524511 805 leader_election.cc:290] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:23.524645 810 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:23.524710 805 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [sys.catalog]: configured and running, proceeding with master startup.
I20250809 14:08:23.524775 810 raft_consensus.cc:695] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [term 1 LEADER]: Becoming Leader. State: Replica: 7c59f412dcbb4a40a67c364f57e57027, State: Running, Role: LEADER
I20250809 14:08:23.524917 810 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c59f412dcbb4a40a67c364f57e57027" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 38861 } }
I20250809 14:08:23.525235 812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c59f412dcbb4a40a67c364f57e57027. Latest consensus state: current_term: 1 leader_uuid: "7c59f412dcbb4a40a67c364f57e57027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c59f412dcbb4a40a67c364f57e57027" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 38861 } } }
I20250809 14:08:23.525238 811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c59f412dcbb4a40a67c364f57e57027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c59f412dcbb4a40a67c364f57e57027" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 38861 } } }
I20250809 14:08:23.525321 812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:23.525327 811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:23.525511 819 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250809 14:08:23.525923 819 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250809 14:08:23.527457 819 catalog_manager.cc:1349] Generated new cluster ID: 89d36707d78448ef898de82db4219786
I20250809 14:08:23.527509 819 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250809 14:08:23.544685 819 catalog_manager.cc:1372] Generated new certificate authority record
I20250809 14:08:23.545431 819 catalog_manager.cc:1506] Loading token signing keys...
I20250809 14:08:23.552119 819 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 7c59f412dcbb4a40a67c364f57e57027: Generated new TSK 0
I20250809 14:08:23.552280 819 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250809 14:08:23.555228 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.193:0
--local_ip_for_outbound_sockets=127.31.21.193
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:38861
--builtin_ntp_servers=127.31.21.212:36131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250809 14:08:23.638818 829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:23.638983 829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:23.639009 829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:23.640316 829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:23.640372 829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.193
I20250809 14:08:23.641722 829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:36131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:38861
--never_fsync=true
--heap_profile_path=/tmp/kudu.829
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.193
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:23.641976 829 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:23.642179 829 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:23.644552 835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:23.644655 837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:23.644719 829 server_base.cc:1047] running on GCE node
W20250809 14:08:23.644586 834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:23.645044 829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:23.645262 829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:23.646427 829 hybrid_clock.cc:648] HybridClock initialized: now 1754748503646410 us; error 31 us; skew 500 ppm
I20250809 14:08:23.647694 829 webserver.cc:489] Webserver started at http://127.31.21.193:38973/ using document root <none> and password file <none>
I20250809 14:08:23.647908 829 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:23.647959 829 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:23.648072 829 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:23.649220 829 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "2517399a9a6e4438b9d1a23909da3f69"
format_stamp: "Formatted at 2025-08-09 14:08:23 on dist-test-slave-mstq"
I20250809 14:08:23.649560 829 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "2517399a9a6e4438b9d1a23909da3f69"
format_stamp: "Formatted at 2025-08-09 14:08:23 on dist-test-slave-mstq"
I20250809 14:08:23.650887 829 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250809 14:08:23.651592 843 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:23.651750 829 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:23.651830 829 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "2517399a9a6e4438b9d1a23909da3f69"
format_stamp: "Formatted at 2025-08-09 14:08:23 on dist-test-slave-mstq"
I20250809 14:08:23.651886 829 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:23.671237 829 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:23.671470 829 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:23.671576 829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:23.671774 829 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:23.672062 829 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:23.672093 829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:23.672127 829 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:23.672151 829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:23.677266 829 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.193:43895
I20250809 14:08:23.677320 957 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.193:43895 every 8 connection(s)
I20250809 14:08:23.677587 829 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250809 14:08:23.679767 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 829
I20250809 14:08:23.679855 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250809 14:08:23.680806 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.194:0
--local_ip_for_outbound_sockets=127.31.21.194
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:38861
--builtin_ntp_servers=127.31.21.212:36131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250809 14:08:23.684793 958 heartbeater.cc:344] Connected to a master server at 127.31.21.254:38861
I20250809 14:08:23.684918 958 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:23.685122 958 heartbeater.cc:507] Master 127.31.21.254:38861 requested a full tablet report, sending...
I20250809 14:08:23.685573 769 ts_manager.cc:194] Registered new tserver with Master: 2517399a9a6e4438b9d1a23909da3f69 (127.31.21.193:43895)
I20250809 14:08:23.686179 769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.193:42221
W20250809 14:08:23.759147 961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:23.759301 961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:23.759320 961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:23.760607 961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:23.760651 961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.194
I20250809 14:08:23.762054 961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:36131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:38861
--never_fsync=true
--heap_profile_path=/tmp/kudu.961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.194
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:23.762238 961 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:23.762420 961 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:23.764786 967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:23.764768 969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:23.764876 966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:23.764962 961 server_base.cc:1047] running on GCE node
I20250809 14:08:23.765168 961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:23.765389 961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:23.766515 961 hybrid_clock.cc:648] HybridClock initialized: now 1754748503766497 us; error 31 us; skew 500 ppm
I20250809 14:08:23.767519 961 webserver.cc:489] Webserver started at http://127.31.21.194:45069/ using document root <none> and password file <none>
I20250809 14:08:23.767692 961 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:23.767733 961 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:23.767848 961 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:23.768644 961 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "068f1c088fac41f4b9d95fce8833684b"
format_stamp: "Formatted at 2025-08-09 14:08:23 on dist-test-slave-mstq"
I20250809 14:08:23.768961 961 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "068f1c088fac41f4b9d95fce8833684b"
format_stamp: "Formatted at 2025-08-09 14:08:23 on dist-test-slave-mstq"
I20250809 14:08:23.770031 961 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20250809 14:08:23.770630 975 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:23.770782 961 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250809 14:08:23.770849 961 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "068f1c088fac41f4b9d95fce8833684b"
format_stamp: "Formatted at 2025-08-09 14:08:23 on dist-test-slave-mstq"
I20250809 14:08:23.770911 961 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:23.779129 961 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:23.779330 961 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:23.779428 961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:23.779605 961 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:23.779871 961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:23.779902 961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:23.779933 961 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:23.779958 961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:23.785116 961 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.194:42825
I20250809 14:08:23.785180 1088 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.194:42825 every 8 connection(s)
I20250809 14:08:23.785437 961 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250809 14:08:23.785778 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 961
I20250809 14:08:23.785868 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250809 14:08:23.787021 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.195:0
--local_ip_for_outbound_sockets=127.31.21.195
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:38861
--builtin_ntp_servers=127.31.21.212:36131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250809 14:08:23.789961 1089 heartbeater.cc:344] Connected to a master server at 127.31.21.254:38861
I20250809 14:08:23.790061 1089 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:23.790293 1089 heartbeater.cc:507] Master 127.31.21.254:38861 requested a full tablet report, sending...
I20250809 14:08:23.790648 769 ts_manager.cc:194] Registered new tserver with Master: 068f1c088fac41f4b9d95fce8833684b (127.31.21.194:42825)
I20250809 14:08:23.791213 769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.194:45283
W20250809 14:08:23.864593 1092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:23.864741 1092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:23.864759 1092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:23.866036 1092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:23.866082 1092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.195
I20250809 14:08:23.867458 1092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:36131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:38861
--never_fsync=true
--heap_profile_path=/tmp/kudu.1092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.195
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:23.867624 1092 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:23.867798 1092 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:23.870196 1098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:23.870249 1097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:23.870196 1100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:23.870374 1092 server_base.cc:1047] running on GCE node
I20250809 14:08:23.870532 1092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:23.870705 1092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:23.871821 1092 hybrid_clock.cc:648] HybridClock initialized: now 1754748503871808 us; error 26 us; skew 500 ppm
I20250809 14:08:23.872871 1092 webserver.cc:489] Webserver started at http://127.31.21.195:38001/ using document root <none> and password file <none>
I20250809 14:08:23.873065 1092 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:23.873118 1092 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:23.873229 1092 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:23.874006 1092 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "41807249277a4adfa9f6df23d86063fd"
format_stamp: "Formatted at 2025-08-09 14:08:23 on dist-test-slave-mstq"
I20250809 14:08:23.874329 1092 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "41807249277a4adfa9f6df23d86063fd"
format_stamp: "Formatted at 2025-08-09 14:08:23 on dist-test-slave-mstq"
I20250809 14:08:23.875430 1092 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20250809 14:08:23.876027 1106 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:23.876173 1092 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:23.876241 1092 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "41807249277a4adfa9f6df23d86063fd"
format_stamp: "Formatted at 2025-08-09 14:08:23 on dist-test-slave-mstq"
I20250809 14:08:23.876292 1092 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:23.885927 1092 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:23.886137 1092 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:23.886240 1092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:23.886412 1092 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:23.886677 1092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:23.886705 1092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:23.886736 1092 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:23.886765 1092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:23.891819 1092 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.195:44853
I20250809 14:08:23.891888 1219 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.195:44853 every 8 connection(s)
I20250809 14:08:23.892134 1092 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250809 14:08:23.895989 1220 heartbeater.cc:344] Connected to a master server at 127.31.21.254:38861
I20250809 14:08:23.896075 1220 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:23.896240 1220 heartbeater.cc:507] Master 127.31.21.254:38861 requested a full tablet report, sending...
I20250809 14:08:23.896538 769 ts_manager.cc:194] Registered new tserver with Master: 41807249277a4adfa9f6df23d86063fd (127.31.21.195:44853)
I20250809 14:08:23.896914 769 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.195:53117
I20250809 14:08:23.901441 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 1092
I20250809 14:08:23.901518 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestDescribeTableColumnFlags.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250809 14:08:23.902383 31831 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250809 14:08:23.907397 769 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44178:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
W20250809 14:08:23.908568 769 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250809 14:08:23.914619 892 tablet_service.cc:1468] Processing CreateTablet for tablet c0f9ef5d7fe1479f9bae0f14d4d30a7e (DEFAULT_TABLE table=TestTable [id=94c9d74984874cbfa14c5372788d35d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:23.914646 1154 tablet_service.cc:1468] Processing CreateTablet for tablet c0f9ef5d7fe1479f9bae0f14d4d30a7e (DEFAULT_TABLE table=TestTable [id=94c9d74984874cbfa14c5372788d35d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:23.914933 1154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f9ef5d7fe1479f9bae0f14d4d30a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:23.914934 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f9ef5d7fe1479f9bae0f14d4d30a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:23.914883 1023 tablet_service.cc:1468] Processing CreateTablet for tablet c0f9ef5d7fe1479f9bae0f14d4d30a7e (DEFAULT_TABLE table=TestTable [id=94c9d74984874cbfa14c5372788d35d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:23.915136 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f9ef5d7fe1479f9bae0f14d4d30a7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:23.916955 1234 tablet_bootstrap.cc:492] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd: Bootstrap starting.
I20250809 14:08:23.917526 1235 tablet_bootstrap.cc:492] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b: Bootstrap starting.
I20250809 14:08:23.917564 1234 tablet_bootstrap.cc:654] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:23.917826 1234 log.cc:826] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:23.917912 1236 tablet_bootstrap.cc:492] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69: Bootstrap starting.
I20250809 14:08:23.918146 1235 tablet_bootstrap.cc:654] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:23.918406 1235 log.cc:826] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:23.918570 1234 tablet_bootstrap.cc:492] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd: No bootstrap required, opened a new log
I20250809 14:08:23.918592 1236 tablet_bootstrap.cc:654] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:23.918789 1234 ts_tablet_manager.cc:1397] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:23.918901 1236 log.cc:826] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:23.919059 1235 tablet_bootstrap.cc:492] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b: No bootstrap required, opened a new log
I20250809 14:08:23.919193 1235 ts_tablet_manager.cc:1397] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:23.919414 1236 tablet_bootstrap.cc:492] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69: No bootstrap required, opened a new log
I20250809 14:08:23.919467 1236 ts_tablet_manager.cc:1397] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:23.920243 1234 raft_consensus.cc:357] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } }
I20250809 14:08:23.920388 1234 raft_consensus.cc:383] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:23.920418 1234 raft_consensus.cc:738] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41807249277a4adfa9f6df23d86063fd, State: Initialized, Role: FOLLOWER
I20250809 14:08:23.920543 1234 consensus_queue.cc:260] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } }
I20250809 14:08:23.920596 1235 raft_consensus.cc:357] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } }
I20250809 14:08:23.920732 1235 raft_consensus.cc:383] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:23.920694 1236 raft_consensus.cc:357] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } }
I20250809 14:08:23.920794 1234 ts_tablet_manager.cc:1428] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:23.920801 1235 raft_consensus.cc:738] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f1c088fac41f4b9d95fce8833684b, State: Initialized, Role: FOLLOWER
I20250809 14:08:23.920830 1236 raft_consensus.cc:383] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:23.920921 1220 heartbeater.cc:499] Master 127.31.21.254:38861 was elected leader, sending a full tablet report...
I20250809 14:08:23.920933 1236 raft_consensus.cc:738] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2517399a9a6e4438b9d1a23909da3f69, State: Initialized, Role: FOLLOWER
I20250809 14:08:23.920929 1235 consensus_queue.cc:260] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } }
I20250809 14:08:23.921044 1236 consensus_queue.cc:260] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } }
I20250809 14:08:23.921198 1235 ts_tablet_manager.cc:1428] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:23.921257 1236 ts_tablet_manager.cc:1428] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:23.921327 1089 heartbeater.cc:499] Master 127.31.21.254:38861 was elected leader, sending a full tablet report...
I20250809 14:08:23.921375 958 heartbeater.cc:499] Master 127.31.21.254:38861 was elected leader, sending a full tablet report...
W20250809 14:08:23.928073 959 tablet.cc:2378] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250809 14:08:24.035988 1090 tablet.cc:2378] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250809 14:08:24.056625 1242 raft_consensus.cc:491] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250809 14:08:24.056738 1242 raft_consensus.cc:513] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } }
I20250809 14:08:24.057086 1242 leader_election.cc:290] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41807249277a4adfa9f6df23d86063fd (127.31.21.195:44853), 068f1c088fac41f4b9d95fce8833684b (127.31.21.194:42825)
I20250809 14:08:24.060114 1174 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0f9ef5d7fe1479f9bae0f14d4d30a7e" candidate_uuid: "2517399a9a6e4438b9d1a23909da3f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41807249277a4adfa9f6df23d86063fd" is_pre_election: true
I20250809 14:08:24.060266 1174 raft_consensus.cc:2466] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2517399a9a6e4438b9d1a23909da3f69 in term 0.
I20250809 14:08:24.060328 1043 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0f9ef5d7fe1479f9bae0f14d4d30a7e" candidate_uuid: "2517399a9a6e4438b9d1a23909da3f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f1c088fac41f4b9d95fce8833684b" is_pre_election: true
I20250809 14:08:24.060461 1043 raft_consensus.cc:2466] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2517399a9a6e4438b9d1a23909da3f69 in term 0.
I20250809 14:08:24.060463 844 leader_election.cc:304] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2517399a9a6e4438b9d1a23909da3f69, 41807249277a4adfa9f6df23d86063fd; no voters:
I20250809 14:08:24.060566 1242 raft_consensus.cc:2802] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250809 14:08:24.060618 1242 raft_consensus.cc:491] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250809 14:08:24.060643 1242 raft_consensus.cc:3058] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:24.061307 1242 raft_consensus.cc:513] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } }
I20250809 14:08:24.061425 1242 leader_election.cc:290] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [CANDIDATE]: Term 1 election: Requested vote from peers 41807249277a4adfa9f6df23d86063fd (127.31.21.195:44853), 068f1c088fac41f4b9d95fce8833684b (127.31.21.194:42825)
I20250809 14:08:24.061563 1174 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0f9ef5d7fe1479f9bae0f14d4d30a7e" candidate_uuid: "2517399a9a6e4438b9d1a23909da3f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41807249277a4adfa9f6df23d86063fd"
I20250809 14:08:24.061554 1043 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0f9ef5d7fe1479f9bae0f14d4d30a7e" candidate_uuid: "2517399a9a6e4438b9d1a23909da3f69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068f1c088fac41f4b9d95fce8833684b"
I20250809 14:08:24.061635 1043 raft_consensus.cc:3058] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:24.061635 1174 raft_consensus.cc:3058] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:24.062407 1043 raft_consensus.cc:2466] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 068f1c088fac41f4b9d95fce8833684b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2517399a9a6e4438b9d1a23909da3f69 in term 1.
I20250809 14:08:24.062407 1174 raft_consensus.cc:2466] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 41807249277a4adfa9f6df23d86063fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2517399a9a6e4438b9d1a23909da3f69 in term 1.
I20250809 14:08:24.062574 846 leader_election.cc:304] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f1c088fac41f4b9d95fce8833684b, 2517399a9a6e4438b9d1a23909da3f69; no voters:
I20250809 14:08:24.062672 1242 raft_consensus.cc:2802] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:24.062822 1242 raft_consensus.cc:695] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [term 1 LEADER]: Becoming Leader. State: Replica: 2517399a9a6e4438b9d1a23909da3f69, State: Running, Role: LEADER
I20250809 14:08:24.063007 1242 consensus_queue.cc:237] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } }
I20250809 14:08:24.063692 769 catalog_manager.cc:5582] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2517399a9a6e4438b9d1a23909da3f69 (127.31.21.193). New cstate: current_term: 1 leader_uuid: "2517399a9a6e4438b9d1a23909da3f69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } health_report { overall_health: UNKNOWN } } }
I20250809 14:08:24.093626 31831 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250809 14:08:24.094039 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 2517399a9a6e4438b9d1a23909da3f69 to finish bootstrapping
I20250809 14:08:24.097188 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 068f1c088fac41f4b9d95fce8833684b to finish bootstrapping
I20250809 14:08:24.100165 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 41807249277a4adfa9f6df23d86063fd to finish bootstrapping
I20250809 14:08:24.103156 769 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44178:
name: "TestAnotherTable"
schema {
columns {
name: "foo"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "bar"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
comment: "comment for bar"
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "foo"
}
}
}
W20250809 14:08:24.103305 769 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestAnotherTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250809 14:08:24.104561 892 tablet_service.cc:1468] Processing CreateTablet for tablet 040b02a3e8eb49239a85e0cfb15262a4 (DEFAULT_TABLE table=TestAnotherTable [id=10a3497503734cf99660d30b03fe7d91]), partition=RANGE (foo) PARTITION UNBOUNDED
I20250809 14:08:24.104561 1023 tablet_service.cc:1468] Processing CreateTablet for tablet 040b02a3e8eb49239a85e0cfb15262a4 (DEFAULT_TABLE table=TestAnotherTable [id=10a3497503734cf99660d30b03fe7d91]), partition=RANGE (foo) PARTITION UNBOUNDED
I20250809 14:08:24.104694 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 040b02a3e8eb49239a85e0cfb15262a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:24.104694 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 040b02a3e8eb49239a85e0cfb15262a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:24.104755 1154 tablet_service.cc:1468] Processing CreateTablet for tablet 040b02a3e8eb49239a85e0cfb15262a4 (DEFAULT_TABLE table=TestAnotherTable [id=10a3497503734cf99660d30b03fe7d91]), partition=RANGE (foo) PARTITION UNBOUNDED
I20250809 14:08:24.104905 1154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 040b02a3e8eb49239a85e0cfb15262a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:24.105773 1234 tablet_bootstrap.cc:492] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd: Bootstrap starting.
I20250809 14:08:24.105929 1236 tablet_bootstrap.cc:492] T 040b02a3e8eb49239a85e0cfb15262a4 P 2517399a9a6e4438b9d1a23909da3f69: Bootstrap starting.
I20250809 14:08:24.105926 1235 tablet_bootstrap.cc:492] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b: Bootstrap starting.
I20250809 14:08:24.106251 1234 tablet_bootstrap.cc:654] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:24.106287 1236 tablet_bootstrap.cc:654] T 040b02a3e8eb49239a85e0cfb15262a4 P 2517399a9a6e4438b9d1a23909da3f69: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:24.106366 1235 tablet_bootstrap.cc:654] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:24.106844 1236 tablet_bootstrap.cc:492] T 040b02a3e8eb49239a85e0cfb15262a4 P 2517399a9a6e4438b9d1a23909da3f69: No bootstrap required, opened a new log
I20250809 14:08:24.106889 1236 ts_tablet_manager.cc:1397] T 040b02a3e8eb49239a85e0cfb15262a4 P 2517399a9a6e4438b9d1a23909da3f69: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:24.106951 1235 tablet_bootstrap.cc:492] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b: No bootstrap required, opened a new log
I20250809 14:08:24.107015 1235 ts_tablet_manager.cc:1397] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:24.107067 1236 raft_consensus.cc:357] T 040b02a3e8eb49239a85e0cfb15262a4 P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } }
I20250809 14:08:24.107136 1236 raft_consensus.cc:383] T 040b02a3e8eb49239a85e0cfb15262a4 P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:24.107158 1236 raft_consensus.cc:738] T 040b02a3e8eb49239a85e0cfb15262a4 P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2517399a9a6e4438b9d1a23909da3f69, State: Initialized, Role: FOLLOWER
I20250809 14:08:24.107168 1235 raft_consensus.cc:357] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } }
I20250809 14:08:24.107227 1235 raft_consensus.cc:383] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:24.107209 1236 consensus_queue.cc:260] T 040b02a3e8eb49239a85e0cfb15262a4 P 2517399a9a6e4438b9d1a23909da3f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } }
I20250809 14:08:24.107254 1235 raft_consensus.cc:738] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068f1c088fac41f4b9d95fce8833684b, State: Initialized, Role: FOLLOWER
I20250809 14:08:24.107300 1236 ts_tablet_manager.cc:1428] T 040b02a3e8eb49239a85e0cfb15262a4 P 2517399a9a6e4438b9d1a23909da3f69: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:24.107295 1235 consensus_queue.cc:260] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } }
I20250809 14:08:24.107385 1235 ts_tablet_manager.cc:1428] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:24.107754 1234 tablet_bootstrap.cc:492] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd: No bootstrap required, opened a new log
I20250809 14:08:24.107801 1234 ts_tablet_manager.cc:1397] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:24.107954 1234 raft_consensus.cc:357] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } }
I20250809 14:08:24.108021 1234 raft_consensus.cc:383] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:24.108049 1234 raft_consensus.cc:738] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41807249277a4adfa9f6df23d86063fd, State: Initialized, Role: FOLLOWER
I20250809 14:08:24.108095 1234 consensus_queue.cc:260] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } }
I20250809 14:08:24.108188 1234 ts_tablet_manager.cc:1428] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20250809 14:08:24.142576 1221 tablet.cc:2378] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250809 14:08:24.339833 1241 raft_consensus.cc:491] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250809 14:08:24.339942 1241 raft_consensus.cc:513] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } }
I20250809 14:08:24.340266 1241 leader_election.cc:290] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2517399a9a6e4438b9d1a23909da3f69 (127.31.21.193:43895), 41807249277a4adfa9f6df23d86063fd (127.31.21.195:44853)
I20250809 14:08:24.342775 1174 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "040b02a3e8eb49239a85e0cfb15262a4" candidate_uuid: "068f1c088fac41f4b9d95fce8833684b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41807249277a4adfa9f6df23d86063fd" is_pre_election: true
I20250809 14:08:24.342792 912 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "040b02a3e8eb49239a85e0cfb15262a4" candidate_uuid: "068f1c088fac41f4b9d95fce8833684b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2517399a9a6e4438b9d1a23909da3f69" is_pre_election: true
I20250809 14:08:24.342859 1174 raft_consensus.cc:2466] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f1c088fac41f4b9d95fce8833684b in term 0.
I20250809 14:08:24.342880 912 raft_consensus.cc:2466] T 040b02a3e8eb49239a85e0cfb15262a4 P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068f1c088fac41f4b9d95fce8833684b in term 0.
I20250809 14:08:24.343036 976 leader_election.cc:304] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f1c088fac41f4b9d95fce8833684b, 41807249277a4adfa9f6df23d86063fd; no voters:
I20250809 14:08:24.343178 1241 raft_consensus.cc:2802] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250809 14:08:24.343235 1241 raft_consensus.cc:491] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250809 14:08:24.343250 1241 raft_consensus.cc:3058] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:24.343839 1241 raft_consensus.cc:513] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } }
I20250809 14:08:24.343951 1241 leader_election.cc:290] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [CANDIDATE]: Term 1 election: Requested vote from peers 2517399a9a6e4438b9d1a23909da3f69 (127.31.21.193:43895), 41807249277a4adfa9f6df23d86063fd (127.31.21.195:44853)
I20250809 14:08:24.344080 912 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "040b02a3e8eb49239a85e0cfb15262a4" candidate_uuid: "068f1c088fac41f4b9d95fce8833684b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2517399a9a6e4438b9d1a23909da3f69"
I20250809 14:08:24.344096 1174 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "040b02a3e8eb49239a85e0cfb15262a4" candidate_uuid: "068f1c088fac41f4b9d95fce8833684b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41807249277a4adfa9f6df23d86063fd"
I20250809 14:08:24.344141 912 raft_consensus.cc:3058] T 040b02a3e8eb49239a85e0cfb15262a4 P 2517399a9a6e4438b9d1a23909da3f69 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:24.344156 1174 raft_consensus.cc:3058] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:24.344686 912 raft_consensus.cc:2466] T 040b02a3e8eb49239a85e0cfb15262a4 P 2517399a9a6e4438b9d1a23909da3f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f1c088fac41f4b9d95fce8833684b in term 1.
I20250809 14:08:24.344728 1174 raft_consensus.cc:2466] T 040b02a3e8eb49239a85e0cfb15262a4 P 41807249277a4adfa9f6df23d86063fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068f1c088fac41f4b9d95fce8833684b in term 1.
I20250809 14:08:24.344870 979 leader_election.cc:304] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068f1c088fac41f4b9d95fce8833684b, 2517399a9a6e4438b9d1a23909da3f69; no voters:
I20250809 14:08:24.344988 1241 raft_consensus.cc:2802] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:24.345139 1241 raft_consensus.cc:695] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [term 1 LEADER]: Becoming Leader. State: Replica: 068f1c088fac41f4b9d95fce8833684b, State: Running, Role: LEADER
I20250809 14:08:24.345228 1241 consensus_queue.cc:237] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } }
I20250809 14:08:24.345947 768 catalog_manager.cc:5582] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b reported cstate change: term changed from 0 to 1, leader changed from <none> to 068f1c088fac41f4b9d95fce8833684b (127.31.21.194). New cstate: current_term: 1 leader_uuid: "068f1c088fac41f4b9d95fce8833684b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 } health_report { overall_health: UNKNOWN } } }
W20250809 14:08:24.460831 1254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:24.461050 1254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:24.462350 1254 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20250809 14:08:24.533344 1242 consensus_queue.cc:1035] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "068f1c088fac41f4b9d95fce8833684b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250809 14:08:24.535104 1244 consensus_queue.cc:1035] T c0f9ef5d7fe1479f9bae0f14d4d30a7e P 2517399a9a6e4438b9d1a23909da3f69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250809 14:08:24.606704 1265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:24.606904 1265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:24.608495 1265 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250809 14:08:24.749519 1280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:24.749675 1280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:24.750993 1280 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20250809 14:08:24.882076 1241 consensus_queue.cc:1035] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [LEADER]: Connected to new peer: Peer: permanent_uuid: "41807249277a4adfa9f6df23d86063fd" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 44853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250809 14:08:24.883932 1293 consensus_queue.cc:1035] T 040b02a3e8eb49239a85e0cfb15262a4 P 068f1c088fac41f4b9d95fce8833684b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2517399a9a6e4438b9d1a23909da3f69" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 43895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250809 14:08:24.895327 1290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:24.895479 1290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:24.896735 1290 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20250809 14:08:24.963294 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 829
I20250809 14:08:24.968140 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 961
I20250809 14:08:24.972384 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 1092
I20250809 14:08:24.976586 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 738
2025-08-09T14:08:24Z chronyd exiting
[ OK ] AdminCliTest.TestDescribeTableColumnFlags (1588 ms)
[ RUN ] AdminCliTest.TestAuthzResetCacheNotAuthorized
I20250809 14:08:24.996330 31831 test_util.cc:276] Using random seed: 961509860
I20250809 14:08:24.996812 31831 ts_itest-base.cc:115] Starting cluster with:
I20250809 14:08:24.996873 31831 ts_itest-base.cc:116] --------------
I20250809 14:08:24.996919 31831 ts_itest-base.cc:117] 3 tablet servers
I20250809 14:08:24.996941 31831 ts_itest-base.cc:118] 3 replicas per TS
I20250809 14:08:24.996963 31831 ts_itest-base.cc:119] --------------
2025-08-09T14:08:24Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-09T14:08:24Z Disabled control of system clock
I20250809 14:08:25.003865 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:41397
--webserver_interface=127.31.21.254
--webserver_port=0
--builtin_ntp_servers=127.31.21.212:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.21.254:41397
--superuser_acl=no-such-user with env {}
W20250809 14:08:25.079322 1311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:25.079489 1311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:25.079522 1311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:25.080868 1311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250809 14:08:25.080909 1311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:25.080933 1311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250809 14:08:25.080955 1311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250809 14:08:25.082473 1311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.21.254:41397
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:41397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.21.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.1311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:25.082721 1311 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:25.082947 1311 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:25.085179 1317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:25.085206 1316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:25.085379 1319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:25.085377 1311 server_base.cc:1047] running on GCE node
I20250809 14:08:25.085625 1311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:25.085846 1311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:25.086995 1311 hybrid_clock.cc:648] HybridClock initialized: now 1754748505086982 us; error 28 us; skew 500 ppm
I20250809 14:08:25.088109 1311 webserver.cc:489] Webserver started at http://127.31.21.254:38533/ using document root <none> and password file <none>
I20250809 14:08:25.088291 1311 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:25.088331 1311 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:25.088443 1311 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:25.089322 1311 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "d732e909ef2a42ecb68fe14fb2081dc6"
format_stamp: "Formatted at 2025-08-09 14:08:25 on dist-test-slave-mstq"
I20250809 14:08:25.089607 1311 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "d732e909ef2a42ecb68fe14fb2081dc6"
format_stamp: "Formatted at 2025-08-09 14:08:25 on dist-test-slave-mstq"
I20250809 14:08:25.090708 1311 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250809 14:08:25.091339 1325 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:25.091471 1311 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:25.091538 1311 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
uuid: "d732e909ef2a42ecb68fe14fb2081dc6"
format_stamp: "Formatted at 2025-08-09 14:08:25 on dist-test-slave-mstq"
I20250809 14:08:25.091595 1311 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:25.107798 1311 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:25.108071 1311 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:25.108188 1311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:25.111815 1311 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.254:41397
I20250809 14:08:25.111886 1377 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.254:41397 every 8 connection(s)
I20250809 14:08:25.112170 1311 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250809 14:08:25.112695 1378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:25.114869 1378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6: Bootstrap starting.
I20250809 14:08:25.115422 1378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:25.115664 1378 log.cc:826] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:25.116322 1378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6: No bootstrap required, opened a new log
I20250809 14:08:25.117439 1378 raft_consensus.cc:357] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d732e909ef2a42ecb68fe14fb2081dc6" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 41397 } }
I20250809 14:08:25.117599 1378 raft_consensus.cc:383] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:25.117621 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 1311
I20250809 14:08:25.117640 1378 raft_consensus.cc:738] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d732e909ef2a42ecb68fe14fb2081dc6, State: Initialized, Role: FOLLOWER
I20250809 14:08:25.117710 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250809 14:08:25.117776 1378 consensus_queue.cc:260] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d732e909ef2a42ecb68fe14fb2081dc6" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 41397 } }
I20250809 14:08:25.117852 1378 raft_consensus.cc:397] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:25.117892 1378 raft_consensus.cc:491] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:25.117950 1378 raft_consensus.cc:3058] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:25.118495 1378 raft_consensus.cc:513] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d732e909ef2a42ecb68fe14fb2081dc6" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 41397 } }
I20250809 14:08:25.118590 1378 leader_election.cc:304] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d732e909ef2a42ecb68fe14fb2081dc6; no voters:
I20250809 14:08:25.118719 1378 leader_election.cc:290] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:25.118778 1383 raft_consensus.cc:2802] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:25.118983 1383 raft_consensus.cc:695] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [term 1 LEADER]: Becoming Leader. State: Replica: d732e909ef2a42ecb68fe14fb2081dc6, State: Running, Role: LEADER
I20250809 14:08:25.119014 1378 sys_catalog.cc:564] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [sys.catalog]: configured and running, proceeding with master startup.
I20250809 14:08:25.119117 1383 consensus_queue.cc:237] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d732e909ef2a42ecb68fe14fb2081dc6" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 41397 } }
I20250809 14:08:25.120108 1384 sys_catalog.cc:455] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d732e909ef2a42ecb68fe14fb2081dc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d732e909ef2a42ecb68fe14fb2081dc6" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 41397 } } }
I20250809 14:08:25.120206 1384 sys_catalog.cc:458] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:25.120381 1395 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250809 14:08:25.120491 1385 sys_catalog.cc:455] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d732e909ef2a42ecb68fe14fb2081dc6. Latest consensus state: current_term: 1 leader_uuid: "d732e909ef2a42ecb68fe14fb2081dc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d732e909ef2a42ecb68fe14fb2081dc6" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 41397 } } }
I20250809 14:08:25.120738 1385 sys_catalog.cc:458] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:25.120949 1395 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250809 14:08:25.122321 1395 catalog_manager.cc:1349] Generated new cluster ID: 1113f95b144543b5abeccc6c9dbe72b3
I20250809 14:08:25.122372 1395 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250809 14:08:25.132615 1395 catalog_manager.cc:1372] Generated new certificate authority record
I20250809 14:08:25.133167 1395 catalog_manager.cc:1506] Loading token signing keys...
I20250809 14:08:25.139256 1395 catalog_manager.cc:5955] T 00000000000000000000000000000000 P d732e909ef2a42ecb68fe14fb2081dc6: Generated new TSK 0
I20250809 14:08:25.139410 1395 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250809 14:08:25.143411 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.193:0
--local_ip_for_outbound_sockets=127.31.21.193
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:41397
--builtin_ntp_servers=127.31.21.212:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250809 14:08:25.220379 1402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:25.220551 1402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:25.220570 1402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:25.222066 1402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:25.222121 1402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.193
I20250809 14:08:25.223531 1402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:41397
--never_fsync=true
--heap_profile_path=/tmp/kudu.1402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.193
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:25.223730 1402 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:25.223950 1402 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:25.226365 1407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:25.226378 1408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:25.226586 1402 server_base.cc:1047] running on GCE node
W20250809 14:08:25.226629 1410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:25.226817 1402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:25.227025 1402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:25.228160 1402 hybrid_clock.cc:648] HybridClock initialized: now 1754748505228146 us; error 33 us; skew 500 ppm
I20250809 14:08:25.229298 1402 webserver.cc:489] Webserver started at http://127.31.21.193:36701/ using document root <none> and password file <none>
I20250809 14:08:25.229507 1402 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:25.229550 1402 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:25.229668 1402 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:25.230515 1402 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "1856db95d3154148ac70a3bdb3312b5a"
format_stamp: "Formatted at 2025-08-09 14:08:25 on dist-test-slave-mstq"
I20250809 14:08:25.230808 1402 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "1856db95d3154148ac70a3bdb3312b5a"
format_stamp: "Formatted at 2025-08-09 14:08:25 on dist-test-slave-mstq"
I20250809 14:08:25.231904 1402 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20250809 14:08:25.232528 1416 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:25.232689 1402 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:25.232759 1402 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "1856db95d3154148ac70a3bdb3312b5a"
format_stamp: "Formatted at 2025-08-09 14:08:25 on dist-test-slave-mstq"
I20250809 14:08:25.232818 1402 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:25.239714 1402 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:25.239916 1402 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:25.240013 1402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:25.240196 1402 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:25.240486 1402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:25.240515 1402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:25.240536 1402 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:25.240562 1402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:25.247272 1402 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.193:42931
I20250809 14:08:25.247367 1529 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.193:42931 every 8 connection(s)
I20250809 14:08:25.247684 1402 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250809 14:08:25.252488 1530 heartbeater.cc:344] Connected to a master server at 127.31.21.254:41397
I20250809 14:08:25.252599 1530 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:25.252795 1530 heartbeater.cc:507] Master 127.31.21.254:41397 requested a full tablet report, sending...
I20250809 14:08:25.253352 1342 ts_manager.cc:194] Registered new tserver with Master: 1856db95d3154148ac70a3bdb3312b5a (127.31.21.193:42931)
I20250809 14:08:25.254313 1342 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.193:58783
I20250809 14:08:25.257578 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 1402
I20250809 14:08:25.257651 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250809 14:08:25.258692 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.194:0
--local_ip_for_outbound_sockets=127.31.21.194
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:41397
--builtin_ntp_servers=127.31.21.212:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250809 14:08:25.340512 1533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:25.340695 1533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:25.340718 1533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:25.342080 1533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:25.342134 1533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.194
I20250809 14:08:25.343580 1533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:41397
--never_fsync=true
--heap_profile_path=/tmp/kudu.1533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.194
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:25.343781 1533 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:25.343988 1533 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:25.346560 1539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:25.346630 1541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:25.346660 1538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:25.346650 1533 server_base.cc:1047] running on GCE node
I20250809 14:08:25.346947 1533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:25.347158 1533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:25.348286 1533 hybrid_clock.cc:648] HybridClock initialized: now 1754748505348267 us; error 36 us; skew 500 ppm
I20250809 14:08:25.349480 1533 webserver.cc:489] Webserver started at http://127.31.21.194:39895/ using document root <none> and password file <none>
I20250809 14:08:25.349689 1533 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:25.349735 1533 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:25.349881 1533 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:25.350739 1533 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "d0f942089ee7453fbc59060207daec4b"
format_stamp: "Formatted at 2025-08-09 14:08:25 on dist-test-slave-mstq"
I20250809 14:08:25.351066 1533 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "d0f942089ee7453fbc59060207daec4b"
format_stamp: "Formatted at 2025-08-09 14:08:25 on dist-test-slave-mstq"
I20250809 14:08:25.352278 1533 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20250809 14:08:25.353026 1547 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:25.353175 1533 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:25.353252 1533 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "d0f942089ee7453fbc59060207daec4b"
format_stamp: "Formatted at 2025-08-09 14:08:25 on dist-test-slave-mstq"
I20250809 14:08:25.353318 1533 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:25.364475 1533 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:25.364743 1533 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:25.364943 1533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:25.365188 1533 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:25.365532 1533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:25.365571 1533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:25.365610 1533 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:25.365644 1533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:25.371572 1533 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.194:34197
I20250809 14:08:25.371641 1661 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.194:34197 every 8 connection(s)
I20250809 14:08:25.371958 1533 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250809 14:08:25.373175 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 1533
I20250809 14:08:25.373247 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250809 14:08:25.374351 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.195:0
--local_ip_for_outbound_sockets=127.31.21.195
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:41397
--builtin_ntp_servers=127.31.21.212:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250809 14:08:25.377010 1662 heartbeater.cc:344] Connected to a master server at 127.31.21.254:41397
I20250809 14:08:25.377104 1662 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:25.377315 1662 heartbeater.cc:507] Master 127.31.21.254:41397 requested a full tablet report, sending...
I20250809 14:08:25.377676 1342 ts_manager.cc:194] Registered new tserver with Master: d0f942089ee7453fbc59060207daec4b (127.31.21.194:34197)
I20250809 14:08:25.378232 1342 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.194:48193
W20250809 14:08:25.452270 1665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:25.452445 1665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:25.452466 1665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:25.453843 1665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:25.453905 1665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.195
I20250809 14:08:25.455309 1665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:41397
--never_fsync=true
--heap_profile_path=/tmp/kudu.1665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.195
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:25.455487 1665 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:25.455688 1665 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:25.458287 1671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:25.458293 1673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:25.458290 1670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:25.458490 1665 server_base.cc:1047] running on GCE node
I20250809 14:08:25.458671 1665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:25.458858 1665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:25.460004 1665 hybrid_clock.cc:648] HybridClock initialized: now 1754748505459987 us; error 36 us; skew 500 ppm
I20250809 14:08:25.461071 1665 webserver.cc:489] Webserver started at http://127.31.21.195:39583/ using document root <none> and password file <none>
I20250809 14:08:25.461239 1665 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:25.461274 1665 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:25.461359 1665 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:25.462152 1665 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "32e332b6d1f4430997475fc6a2b77b20"
format_stamp: "Formatted at 2025-08-09 14:08:25 on dist-test-slave-mstq"
I20250809 14:08:25.462414 1665 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "32e332b6d1f4430997475fc6a2b77b20"
format_stamp: "Formatted at 2025-08-09 14:08:25 on dist-test-slave-mstq"
I20250809 14:08:25.463500 1665 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20250809 14:08:25.464179 1679 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:25.464357 1665 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:25.464432 1665 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "32e332b6d1f4430997475fc6a2b77b20"
format_stamp: "Formatted at 2025-08-09 14:08:25 on dist-test-slave-mstq"
I20250809 14:08:25.464486 1665 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:25.477077 1665 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:25.477301 1665 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:25.477393 1665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:25.477586 1665 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:25.477864 1665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:25.477896 1665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:25.477927 1665 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:25.477947 1665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:25.483690 1665 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.195:35651
I20250809 14:08:25.483764 1792 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.195:35651 every 8 connection(s)
I20250809 14:08:25.484051 1665 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250809 14:08:25.488615 1793 heartbeater.cc:344] Connected to a master server at 127.31.21.254:41397
I20250809 14:08:25.488742 1793 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:25.488821 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 1665
I20250809 14:08:25.488935 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestAuthzResetCacheNotAuthorized.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250809 14:08:25.489034 1793 heartbeater.cc:507] Master 127.31.21.254:41397 requested a full tablet report, sending...
I20250809 14:08:25.489427 1342 ts_manager.cc:194] Registered new tserver with Master: 32e332b6d1f4430997475fc6a2b77b20 (127.31.21.195:35651)
I20250809 14:08:25.489874 31831 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250809 14:08:25.489869 1342 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.195:49655
I20250809 14:08:25.495210 1342 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48904:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
W20250809 14:08:25.496419 1342 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250809 14:08:25.502487 1727 tablet_service.cc:1468] Processing CreateTablet for tablet 8d4e3b16cd1340a4a0dab56fab48f4f5 (DEFAULT_TABLE table=TestTable [id=db683c41b4dd40ebae8172eb933c391f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:25.502489 1596 tablet_service.cc:1468] Processing CreateTablet for tablet 8d4e3b16cd1340a4a0dab56fab48f4f5 (DEFAULT_TABLE table=TestTable [id=db683c41b4dd40ebae8172eb933c391f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:25.502766 1596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d4e3b16cd1340a4a0dab56fab48f4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:25.502772 1727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d4e3b16cd1340a4a0dab56fab48f4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:25.504773 1807 tablet_bootstrap.cc:492] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b: Bootstrap starting.
I20250809 14:08:25.505000 1808 tablet_bootstrap.cc:492] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20: Bootstrap starting.
I20250809 14:08:25.505620 1808 tablet_bootstrap.cc:654] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:25.505788 1464 tablet_service.cc:1468] Processing CreateTablet for tablet 8d4e3b16cd1340a4a0dab56fab48f4f5 (DEFAULT_TABLE table=TestTable [id=db683c41b4dd40ebae8172eb933c391f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:25.505898 1808 log.cc:826] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:25.506079 1464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d4e3b16cd1340a4a0dab56fab48f4f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:25.506469 1808 tablet_bootstrap.cc:492] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20: No bootstrap required, opened a new log
I20250809 14:08:25.506527 1808 ts_tablet_manager.cc:1397] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:25.506651 1807 tablet_bootstrap.cc:654] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:25.506925 1807 log.cc:826] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:25.507565 1807 tablet_bootstrap.cc:492] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b: No bootstrap required, opened a new log
I20250809 14:08:25.507674 1807 ts_tablet_manager.cc:1397] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250809 14:08:25.507915 1808 raft_consensus.cc:357] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1856db95d3154148ac70a3bdb3312b5a" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 42931 } } peers { permanent_uuid: "32e332b6d1f4430997475fc6a2b77b20" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 35651 } } peers { permanent_uuid: "d0f942089ee7453fbc59060207daec4b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34197 } }
I20250809 14:08:25.508033 1808 raft_consensus.cc:383] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:25.508061 1808 raft_consensus.cc:738] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32e332b6d1f4430997475fc6a2b77b20, State: Initialized, Role: FOLLOWER
I20250809 14:08:25.508176 1808 consensus_queue.cc:260] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1856db95d3154148ac70a3bdb3312b5a" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 42931 } } peers { permanent_uuid: "32e332b6d1f4430997475fc6a2b77b20" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 35651 } } peers { permanent_uuid: "d0f942089ee7453fbc59060207daec4b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34197 } }
I20250809 14:08:25.508399 1808 ts_tablet_manager.cc:1428] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:25.508510 1793 heartbeater.cc:499] Master 127.31.21.254:41397 was elected leader, sending a full tablet report...
I20250809 14:08:25.508662 1812 tablet_bootstrap.cc:492] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a: Bootstrap starting.
I20250809 14:08:25.509327 1812 tablet_bootstrap.cc:654] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:25.509469 1807 raft_consensus.cc:357] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1856db95d3154148ac70a3bdb3312b5a" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 42931 } } peers { permanent_uuid: "32e332b6d1f4430997475fc6a2b77b20" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 35651 } } peers { permanent_uuid: "d0f942089ee7453fbc59060207daec4b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34197 } }
I20250809 14:08:25.509601 1807 raft_consensus.cc:383] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:25.509595 1812 log.cc:826] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:25.509686 1807 raft_consensus.cc:738] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f942089ee7453fbc59060207daec4b, State: Initialized, Role: FOLLOWER
I20250809 14:08:25.509785 1807 consensus_queue.cc:260] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1856db95d3154148ac70a3bdb3312b5a" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 42931 } } peers { permanent_uuid: "32e332b6d1f4430997475fc6a2b77b20" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 35651 } } peers { permanent_uuid: "d0f942089ee7453fbc59060207daec4b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34197 } }
I20250809 14:08:25.510004 1807 ts_tablet_manager.cc:1428] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:25.510115 1662 heartbeater.cc:499] Master 127.31.21.254:41397 was elected leader, sending a full tablet report...
I20250809 14:08:25.510149 1812 tablet_bootstrap.cc:492] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a: No bootstrap required, opened a new log
I20250809 14:08:25.510203 1812 ts_tablet_manager.cc:1397] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:25.511528 1812 raft_consensus.cc:357] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1856db95d3154148ac70a3bdb3312b5a" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 42931 } } peers { permanent_uuid: "32e332b6d1f4430997475fc6a2b77b20" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 35651 } } peers { permanent_uuid: "d0f942089ee7453fbc59060207daec4b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34197 } }
I20250809 14:08:25.511628 1812 raft_consensus.cc:383] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:25.511656 1812 raft_consensus.cc:738] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1856db95d3154148ac70a3bdb3312b5a, State: Initialized, Role: FOLLOWER
I20250809 14:08:25.511740 1812 consensus_queue.cc:260] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1856db95d3154148ac70a3bdb3312b5a" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 42931 } } peers { permanent_uuid: "32e332b6d1f4430997475fc6a2b77b20" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 35651 } } peers { permanent_uuid: "d0f942089ee7453fbc59060207daec4b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34197 } }
I20250809 14:08:25.511938 1812 ts_tablet_manager.cc:1428] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:25.511986 1530 heartbeater.cc:499] Master 127.31.21.254:41397 was elected leader, sending a full tablet report...
W20250809 14:08:25.622434 1663 tablet.cc:2378] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250809 14:08:25.672461 1814 raft_consensus.cc:491] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250809 14:08:25.672593 1814 raft_consensus.cc:513] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1856db95d3154148ac70a3bdb3312b5a" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 42931 } } peers { permanent_uuid: "32e332b6d1f4430997475fc6a2b77b20" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 35651 } } peers { permanent_uuid: "d0f942089ee7453fbc59060207daec4b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34197 } }
I20250809 14:08:25.672940 1814 leader_election.cc:290] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1856db95d3154148ac70a3bdb3312b5a (127.31.21.193:42931), 32e332b6d1f4430997475fc6a2b77b20 (127.31.21.195:35651)
I20250809 14:08:25.676267 1747 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8d4e3b16cd1340a4a0dab56fab48f4f5" candidate_uuid: "d0f942089ee7453fbc59060207daec4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32e332b6d1f4430997475fc6a2b77b20" is_pre_election: true
I20250809 14:08:25.676350 1484 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8d4e3b16cd1340a4a0dab56fab48f4f5" candidate_uuid: "d0f942089ee7453fbc59060207daec4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1856db95d3154148ac70a3bdb3312b5a" is_pre_election: true
I20250809 14:08:25.676440 1747 raft_consensus.cc:2466] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f942089ee7453fbc59060207daec4b in term 0.
I20250809 14:08:25.676462 1484 raft_consensus.cc:2466] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0f942089ee7453fbc59060207daec4b in term 0.
I20250809 14:08:25.676674 1549 leader_election.cc:304] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32e332b6d1f4430997475fc6a2b77b20, d0f942089ee7453fbc59060207daec4b; no voters:
I20250809 14:08:25.676805 1814 raft_consensus.cc:2802] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250809 14:08:25.676892 1814 raft_consensus.cc:491] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250809 14:08:25.676930 1814 raft_consensus.cc:3058] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:25.677556 1814 raft_consensus.cc:513] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1856db95d3154148ac70a3bdb3312b5a" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 42931 } } peers { permanent_uuid: "32e332b6d1f4430997475fc6a2b77b20" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 35651 } } peers { permanent_uuid: "d0f942089ee7453fbc59060207daec4b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34197 } }
I20250809 14:08:25.677680 1814 leader_election.cc:290] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [CANDIDATE]: Term 1 election: Requested vote from peers 1856db95d3154148ac70a3bdb3312b5a (127.31.21.193:42931), 32e332b6d1f4430997475fc6a2b77b20 (127.31.21.195:35651)
I20250809 14:08:25.677814 1484 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8d4e3b16cd1340a4a0dab56fab48f4f5" candidate_uuid: "d0f942089ee7453fbc59060207daec4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1856db95d3154148ac70a3bdb3312b5a"
I20250809 14:08:25.677819 1747 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8d4e3b16cd1340a4a0dab56fab48f4f5" candidate_uuid: "d0f942089ee7453fbc59060207daec4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32e332b6d1f4430997475fc6a2b77b20"
I20250809 14:08:25.677883 1747 raft_consensus.cc:3058] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:25.677883 1484 raft_consensus.cc:3058] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:25.678469 1484 raft_consensus.cc:2466] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f942089ee7453fbc59060207daec4b in term 1.
I20250809 14:08:25.678509 1747 raft_consensus.cc:2466] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0f942089ee7453fbc59060207daec4b in term 1.
I20250809 14:08:25.678635 1551 leader_election.cc:304] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1856db95d3154148ac70a3bdb3312b5a, d0f942089ee7453fbc59060207daec4b; no voters:
I20250809 14:08:25.678732 1814 raft_consensus.cc:2802] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:25.678877 1814 raft_consensus.cc:695] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [term 1 LEADER]: Becoming Leader. State: Replica: d0f942089ee7453fbc59060207daec4b, State: Running, Role: LEADER
I20250809 14:08:25.679011 1814 consensus_queue.cc:237] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1856db95d3154148ac70a3bdb3312b5a" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 42931 } } peers { permanent_uuid: "32e332b6d1f4430997475fc6a2b77b20" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 35651 } } peers { permanent_uuid: "d0f942089ee7453fbc59060207daec4b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34197 } }
I20250809 14:08:25.679736 1342 catalog_manager.cc:5582] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P d0f942089ee7453fbc59060207daec4b reported cstate change: term changed from 0 to 1, leader changed from <none> to d0f942089ee7453fbc59060207daec4b (127.31.21.194). New cstate: current_term: 1 leader_uuid: "d0f942089ee7453fbc59060207daec4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1856db95d3154148ac70a3bdb3312b5a" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 42931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32e332b6d1f4430997475fc6a2b77b20" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 35651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0f942089ee7453fbc59060207daec4b" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34197 } health_report { overall_health: HEALTHY } } }
I20250809 14:08:25.726682 31831 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250809 14:08:25.727101 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 1856db95d3154148ac70a3bdb3312b5a to finish bootstrapping
I20250809 14:08:25.730489 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver d0f942089ee7453fbc59060207daec4b to finish bootstrapping
I20250809 14:08:25.733426 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 32e332b6d1f4430997475fc6a2b77b20 to finish bootstrapping
W20250809 14:08:25.734570 1794 tablet.cc:2378] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 32e332b6d1f4430997475fc6a2b77b20: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250809 14:08:25.748402 1531 tablet.cc:2378] T 8d4e3b16cd1340a4a0dab56fab48f4f5 P 1856db95d3154148ac70a3bdb3312b5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250809 14:08:25.827061 1342 server_base.cc:1129] Unauthorized access attempt to method kudu.master.MasterService.RefreshAuthzCache from {username='slave'} at 127.0.0.1:48912
I20250809 14:08:25.882711 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 1402
I20250809 14:08:25.887272 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 1533
I20250809 14:08:25.891525 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 1665
I20250809 14:08:25.895843 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 1311
2025-08-09T14:08:25Z chronyd exiting
[ OK ] AdminCliTest.TestAuthzResetCacheNotAuthorized (918 ms)
[ RUN ] AdminCliTest.TestRebuildTables
I20250809 14:08:25.915266 31831 test_util.cc:276] Using random seed: 962428796
I20250809 14:08:25.915762 31831 ts_itest-base.cc:115] Starting cluster with:
I20250809 14:08:25.915799 31831 ts_itest-base.cc:116] --------------
I20250809 14:08:25.915810 31831 ts_itest-base.cc:117] 3 tablet servers
I20250809 14:08:25.915822 31831 ts_itest-base.cc:118] 3 replicas per TS
I20250809 14:08:25.915851 31831 ts_itest-base.cc:119] --------------
2025-08-09T14:08:25Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-09T14:08:25Z Disabled control of system clock
I20250809 14:08:25.922709 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:37189
--webserver_interface=127.31.21.254
--webserver_port=0
--builtin_ntp_servers=127.31.21.212:43147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.21.254:37189 with env {}
W20250809 14:08:25.998687 1842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:25.998873 1842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:25.998906 1842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:26.000293 1842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250809 14:08:26.000344 1842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:26.000365 1842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250809 14:08:26.000385 1842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250809 14:08:26.001864 1842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:43147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.21.254:37189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:37189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.21.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.1842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:26.002111 1842 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:26.002329 1842 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:26.004752 1848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:26.004755 1847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:26.004871 1842 server_base.cc:1047] running on GCE node
W20250809 14:08:26.004767 1850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:26.005162 1842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:26.005363 1842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:26.006491 1842 hybrid_clock.cc:648] HybridClock initialized: now 1754748506006474 us; error 31 us; skew 500 ppm
I20250809 14:08:26.007530 1842 webserver.cc:489] Webserver started at http://127.31.21.254:34899/ using document root <none> and password file <none>
I20250809 14:08:26.007717 1842 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:26.007755 1842 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:26.007872 1842 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:26.008705 1842 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "dd5eb26ca1e742f2a81372fea15d8b56"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:26.009016 1842 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "dd5eb26ca1e742f2a81372fea15d8b56"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:26.010128 1842 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20250809 14:08:26.010767 1856 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:26.010905 1842 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250809 14:08:26.010972 1842 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
uuid: "dd5eb26ca1e742f2a81372fea15d8b56"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:26.011023 1842 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:26.028232 1842 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:26.028517 1842 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:26.028623 1842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:26.032194 1842 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.254:37189
I20250809 14:08:26.032260 1909 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.254:37189 every 8 connection(s)
I20250809 14:08:26.032542 1842 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250809 14:08:26.033115 1910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:26.035320 1910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Bootstrap starting.
I20250809 14:08:26.035892 1910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:26.036157 1910 log.cc:826] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:26.036409 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 1842
I20250809 14:08:26.036499 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250809 14:08:26.036736 1910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: No bootstrap required, opened a new log
I20250809 14:08:26.038391 1910 raft_consensus.cc:357] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } }
I20250809 14:08:26.038535 1910 raft_consensus.cc:383] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:26.038563 1910 raft_consensus.cc:738] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd5eb26ca1e742f2a81372fea15d8b56, State: Initialized, Role: FOLLOWER
I20250809 14:08:26.038656 1910 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } }
I20250809 14:08:26.038714 1910 raft_consensus.cc:397] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:26.038748 1910 raft_consensus.cc:491] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:26.038791 1910 raft_consensus.cc:3058] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:26.039388 1910 raft_consensus.cc:513] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } }
I20250809 14:08:26.039502 1910 leader_election.cc:304] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd5eb26ca1e742f2a81372fea15d8b56; no voters:
I20250809 14:08:26.039652 1910 leader_election.cc:290] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:26.039701 1915 raft_consensus.cc:2802] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:26.039855 1915 raft_consensus.cc:695] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 1 LEADER]: Becoming Leader. State: Replica: dd5eb26ca1e742f2a81372fea15d8b56, State: Running, Role: LEADER
I20250809 14:08:26.039865 1910 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: configured and running, proceeding with master startup.
I20250809 14:08:26.040000 1915 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } }
I20250809 14:08:26.040304 1916 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } } }
I20250809 14:08:26.040313 1917 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd5eb26ca1e742f2a81372fea15d8b56. Latest consensus state: current_term: 1 leader_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } } }
I20250809 14:08:26.040396 1916 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:26.040400 1917 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:26.040676 1919 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250809 14:08:26.041208 1919 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250809 14:08:26.042840 1919 catalog_manager.cc:1349] Generated new cluster ID: e9f2666a32674795877a734dc9a3e6b0
I20250809 14:08:26.042905 1919 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250809 14:08:26.054699 1919 catalog_manager.cc:1372] Generated new certificate authority record
I20250809 14:08:26.055219 1919 catalog_manager.cc:1506] Loading token signing keys...
I20250809 14:08:26.058266 1919 catalog_manager.cc:5955] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Generated new TSK 0
I20250809 14:08:26.058434 1919 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250809 14:08:26.062971 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.193:0
--local_ip_for_outbound_sockets=127.31.21.193
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:37189
--builtin_ntp_servers=127.31.21.212:43147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250809 14:08:26.138626 1935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:26.138797 1935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:26.138830 1935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:26.140210 1935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:26.140265 1935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.193
I20250809 14:08:26.141916 1935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:43147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:37189
--never_fsync=true
--heap_profile_path=/tmp/kudu.1935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.193
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:26.142160 1935 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:26.142374 1935 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:26.144800 1940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:26.144780 1941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:26.144958 1943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:26.144992 1935 server_base.cc:1047] running on GCE node
I20250809 14:08:26.145273 1935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:26.145462 1935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:26.146592 1935 hybrid_clock.cc:648] HybridClock initialized: now 1754748506146578 us; error 30 us; skew 500 ppm
I20250809 14:08:26.147720 1935 webserver.cc:489] Webserver started at http://127.31.21.193:41677/ using document root <none> and password file <none>
I20250809 14:08:26.147915 1935 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:26.147956 1935 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:26.148069 1935 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:26.148905 1935 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "5a52eadefc7043b7bc9cc388cb50cb4d"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:26.149227 1935 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "5a52eadefc7043b7bc9cc388cb50cb4d"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:26.150430 1935 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20250809 14:08:26.151124 1949 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:26.151285 1935 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250809 14:08:26.151355 1935 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "5a52eadefc7043b7bc9cc388cb50cb4d"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:26.151412 1935 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:26.171833 1935 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:26.172089 1935 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:26.172194 1935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:26.172412 1935 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:26.172719 1935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:26.172751 1935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:26.172787 1935 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:26.172812 1935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:26.178664 1935 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.193:40613
I20250809 14:08:26.178758 2062 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.193:40613 every 8 connection(s)
I20250809 14:08:26.179024 1935 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250809 14:08:26.183796 2063 heartbeater.cc:344] Connected to a master server at 127.31.21.254:37189
I20250809 14:08:26.183892 2063 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:26.184083 2063 heartbeater.cc:507] Master 127.31.21.254:37189 requested a full tablet report, sending...
I20250809 14:08:26.184512 1873 ts_manager.cc:194] Registered new tserver with Master: 5a52eadefc7043b7bc9cc388cb50cb4d (127.31.21.193:40613)
I20250809 14:08:26.185528 1873 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.193:41307
I20250809 14:08:26.187251 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 1935
I20250809 14:08:26.187320 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250809 14:08:26.188289 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.194:0
--local_ip_for_outbound_sockets=127.31.21.194
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:37189
--builtin_ntp_servers=127.31.21.212:43147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250809 14:08:26.262699 2066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:26.262856 2066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:26.262876 2066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:26.264222 2066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:26.264271 2066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.194
I20250809 14:08:26.265723 2066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:43147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:37189
--never_fsync=true
--heap_profile_path=/tmp/kudu.2066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.194
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:26.265916 2066 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:26.266134 2066 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:26.268447 2074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:26.268525 2071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:26.268440 2072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:26.268599 2066 server_base.cc:1047] running on GCE node
I20250809 14:08:26.268817 2066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:26.269026 2066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:26.270153 2066 hybrid_clock.cc:648] HybridClock initialized: now 1754748506270141 us; error 28 us; skew 500 ppm
I20250809 14:08:26.271178 2066 webserver.cc:489] Webserver started at http://127.31.21.194:33809/ using document root <none> and password file <none>
I20250809 14:08:26.271363 2066 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:26.271406 2066 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:26.271520 2066 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:26.272365 2066 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "b7dd44e3fbd744d39cc8a3c3307b3940"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:26.272644 2066 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "b7dd44e3fbd744d39cc8a3c3307b3940"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:26.273800 2066 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250809 14:08:26.274494 2080 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:26.274655 2066 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:26.274724 2066 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "b7dd44e3fbd744d39cc8a3c3307b3940"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:26.274780 2066 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:26.297561 2066 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:26.297830 2066 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:26.297958 2066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:26.298182 2066 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:26.298487 2066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:26.298520 2066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:26.298557 2066 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:26.298599 2066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:26.304234 2066 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.194:46267
I20250809 14:08:26.304287 2193 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.194:46267 every 8 connection(s)
I20250809 14:08:26.304562 2066 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250809 14:08:26.308518 2194 heartbeater.cc:344] Connected to a master server at 127.31.21.254:37189
I20250809 14:08:26.308599 2194 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:26.308751 2194 heartbeater.cc:507] Master 127.31.21.254:37189 requested a full tablet report, sending...
I20250809 14:08:26.309100 1873 ts_manager.cc:194] Registered new tserver with Master: b7dd44e3fbd744d39cc8a3c3307b3940 (127.31.21.194:46267)
I20250809 14:08:26.309449 1873 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.194:58209
I20250809 14:08:26.312469 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 2066
I20250809 14:08:26.312544 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250809 14:08:26.313457 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.195:0
--local_ip_for_outbound_sockets=127.31.21.195
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:37189
--builtin_ntp_servers=127.31.21.212:43147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250809 14:08:26.389668 2197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:26.389866 2197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:26.389885 2197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:26.391238 2197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:26.391283 2197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.195
I20250809 14:08:26.392674 2197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:43147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:37189
--never_fsync=true
--heap_profile_path=/tmp/kudu.2197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.195
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:26.392913 2197 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:26.393121 2197 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:26.395563 2205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:26.395632 2197 server_base.cc:1047] running on GCE node
W20250809 14:08:26.395622 2203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:26.395563 2202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:26.395882 2197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:26.396061 2197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:26.397177 2197 hybrid_clock.cc:648] HybridClock initialized: now 1754748506397171 us; error 31 us; skew 500 ppm
I20250809 14:08:26.398300 2197 webserver.cc:489] Webserver started at http://127.31.21.195:39965/ using document root <none> and password file <none>
I20250809 14:08:26.398486 2197 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:26.398528 2197 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:26.398626 2197 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:26.399436 2197 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "4af13b22c6ad44cf94062caee49dc81b"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:26.399720 2197 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "4af13b22c6ad44cf94062caee49dc81b"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:26.400844 2197 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20250809 14:08:26.401604 2211 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:26.401759 2197 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:26.401823 2197 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "4af13b22c6ad44cf94062caee49dc81b"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:26.401883 2197 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:26.417330 2197 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:26.417630 2197 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:26.417740 2197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:26.418027 2197 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:26.418486 2197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:26.418529 2197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:26.418566 2197 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:26.418591 2197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:26.424026 2197 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.195:45239
I20250809 14:08:26.424077 2324 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.195:45239 every 8 connection(s)
I20250809 14:08:26.424373 2197 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250809 14:08:26.427362 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 2197
I20250809 14:08:26.427451 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250809 14:08:26.428682 2325 heartbeater.cc:344] Connected to a master server at 127.31.21.254:37189
I20250809 14:08:26.428777 2325 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:26.428984 2325 heartbeater.cc:507] Master 127.31.21.254:37189 requested a full tablet report, sending...
I20250809 14:08:26.429302 1873 ts_manager.cc:194] Registered new tserver with Master: 4af13b22c6ad44cf94062caee49dc81b (127.31.21.195:45239)
I20250809 14:08:26.429656 1873 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.195:34395
I20250809 14:08:26.439352 31831 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250809 14:08:26.445175 31831 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250809 14:08:26.445242 31831 test_util.cc:276] Using random seed: 962958779
I20250809 14:08:26.450567 1873 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59546:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250809 14:08:26.456056 2128 tablet_service.cc:1468] Processing CreateTablet for tablet e035914df64a44b3884a91c9524340d0 (DEFAULT_TABLE table=TestTable [id=962c032326214b0e81a431a74f08c386]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:26.456337 2128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e035914df64a44b3884a91c9524340d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:26.458542 2340 tablet_bootstrap.cc:492] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: Bootstrap starting.
I20250809 14:08:26.459069 2340 tablet_bootstrap.cc:654] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:26.459298 2340 log.cc:826] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:26.459825 2340 tablet_bootstrap.cc:492] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: No bootstrap required, opened a new log
I20250809 14:08:26.459880 2340 ts_tablet_manager.cc:1397] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:26.461120 2340 raft_consensus.cc:357] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } }
I20250809 14:08:26.461236 2340 raft_consensus.cc:383] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:26.461266 2340 raft_consensus.cc:738] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7dd44e3fbd744d39cc8a3c3307b3940, State: Initialized, Role: FOLLOWER
I20250809 14:08:26.461364 2340 consensus_queue.cc:260] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } }
I20250809 14:08:26.461419 2340 raft_consensus.cc:397] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:26.461472 2340 raft_consensus.cc:491] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:26.461516 2340 raft_consensus.cc:3058] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:26.462003 2340 raft_consensus.cc:513] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } }
I20250809 14:08:26.462100 2340 leader_election.cc:304] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7dd44e3fbd744d39cc8a3c3307b3940; no voters:
I20250809 14:08:26.462227 2340 leader_election.cc:290] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:26.462282 2342 raft_consensus.cc:2802] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:26.462407 2340 ts_tablet_manager.cc:1428] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250809 14:08:26.462442 2342 raft_consensus.cc:695] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 1 LEADER]: Becoming Leader. State: Replica: b7dd44e3fbd744d39cc8a3c3307b3940, State: Running, Role: LEADER
I20250809 14:08:26.462428 2194 heartbeater.cc:499] Master 127.31.21.254:37189 was elected leader, sending a full tablet report...
I20250809 14:08:26.462633 2342 consensus_queue.cc:237] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } }
I20250809 14:08:26.463372 1873 catalog_manager.cc:5582] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7dd44e3fbd744d39cc8a3c3307b3940 (127.31.21.194). New cstate: current_term: 1 leader_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } health_report { overall_health: HEALTHY } } }
I20250809 14:08:26.475968 31831 test_util.cc:276] Using random seed: 962989494
I20250809 14:08:26.480146 1873 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59558:
name: "TestTable1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250809 14:08:26.484858 2259 tablet_service.cc:1468] Processing CreateTablet for tablet f626b930b2c549dfaff430564d98a8fa (DEFAULT_TABLE table=TestTable1 [id=d5212020585f41f48131e8caa6b41e87]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:26.485172 2259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f626b930b2c549dfaff430564d98a8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:26.486986 2361 tablet_bootstrap.cc:492] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: Bootstrap starting.
I20250809 14:08:26.487577 2361 tablet_bootstrap.cc:654] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:26.487811 2361 log.cc:826] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:26.488373 2361 tablet_bootstrap.cc:492] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: No bootstrap required, opened a new log
I20250809 14:08:26.488421 2361 ts_tablet_manager.cc:1397] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:26.489483 2361 raft_consensus.cc:357] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } }
I20250809 14:08:26.489586 2361 raft_consensus.cc:383] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:26.489605 2361 raft_consensus.cc:738] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af13b22c6ad44cf94062caee49dc81b, State: Initialized, Role: FOLLOWER
I20250809 14:08:26.489693 2361 consensus_queue.cc:260] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } }
I20250809 14:08:26.489741 2361 raft_consensus.cc:397] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:26.489775 2361 raft_consensus.cc:491] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:26.489820 2361 raft_consensus.cc:3058] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:26.490377 2361 raft_consensus.cc:513] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } }
I20250809 14:08:26.490468 2361 leader_election.cc:304] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4af13b22c6ad44cf94062caee49dc81b; no voters:
I20250809 14:08:26.490607 2361 leader_election.cc:290] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:26.490649 2363 raft_consensus.cc:2802] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:26.490777 2363 raft_consensus.cc:695] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 1 LEADER]: Becoming Leader. State: Replica: 4af13b22c6ad44cf94062caee49dc81b, State: Running, Role: LEADER
I20250809 14:08:26.490824 2325 heartbeater.cc:499] Master 127.31.21.254:37189 was elected leader, sending a full tablet report...
I20250809 14:08:26.490886 2363 consensus_queue.cc:237] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } }
I20250809 14:08:26.490798 2361 ts_tablet_manager.cc:1428] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250809 14:08:26.491501 1873 catalog_manager.cc:5582] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4af13b22c6ad44cf94062caee49dc81b (127.31.21.195). New cstate: current_term: 1 leader_uuid: "4af13b22c6ad44cf94062caee49dc81b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } health_report { overall_health: HEALTHY } } }
I20250809 14:08:26.504688 31831 test_util.cc:276] Using random seed: 963018215
I20250809 14:08:26.509112 1873 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59572:
name: "TestTable2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250809 14:08:26.513840 1997 tablet_service.cc:1468] Processing CreateTablet for tablet 8fe312fd225e401db30ae621e081b279 (DEFAULT_TABLE table=TestTable2 [id=381ab3452c924286bc13e0224537174f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:26.514106 1997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fe312fd225e401db30ae621e081b279. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:26.516167 2381 tablet_bootstrap.cc:492] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d: Bootstrap starting.
I20250809 14:08:26.516722 2381 tablet_bootstrap.cc:654] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:26.516978 2381 log.cc:826] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:26.517493 2381 tablet_bootstrap.cc:492] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d: No bootstrap required, opened a new log
I20250809 14:08:26.517539 2381 ts_tablet_manager.cc:1397] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20250809 14:08:26.518491 2381 raft_consensus.cc:357] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 40613 } }
I20250809 14:08:26.518602 2381 raft_consensus.cc:383] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:26.518642 2381 raft_consensus.cc:738] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a52eadefc7043b7bc9cc388cb50cb4d, State: Initialized, Role: FOLLOWER
I20250809 14:08:26.518774 2381 consensus_queue.cc:260] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 40613 } }
I20250809 14:08:26.518857 2381 raft_consensus.cc:397] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:26.518899 2381 raft_consensus.cc:491] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:26.518940 2381 raft_consensus.cc:3058] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:26.519404 2381 raft_consensus.cc:513] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 40613 } }
I20250809 14:08:26.519481 2381 leader_election.cc:304] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a52eadefc7043b7bc9cc388cb50cb4d; no voters:
I20250809 14:08:26.519590 2381 leader_election.cc:290] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:26.519642 2383 raft_consensus.cc:2802] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:26.519737 2381 ts_tablet_manager.cc:1428] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250809 14:08:26.519770 2063 heartbeater.cc:499] Master 127.31.21.254:37189 was elected leader, sending a full tablet report...
I20250809 14:08:26.519794 2383 raft_consensus.cc:695] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 1 LEADER]: Becoming Leader. State: Replica: 5a52eadefc7043b7bc9cc388cb50cb4d, State: Running, Role: LEADER
I20250809 14:08:26.519910 2383 consensus_queue.cc:237] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 40613 } }
I20250809 14:08:26.520532 1873 catalog_manager.cc:5582] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a52eadefc7043b7bc9cc388cb50cb4d (127.31.21.193). New cstate: current_term: 1 leader_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 40613 } health_report { overall_health: HEALTHY } } }
I20250809 14:08:26.533424 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 1842
I20250809 14:08:26.908504 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 1935
I20250809 14:08:26.912884 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 2066
I20250809 14:08:26.917038 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 2197
I20250809 14:08:26.921263 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:37189
--webserver_interface=127.31.21.254
--webserver_port=34899
--builtin_ntp_servers=127.31.21.212:43147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.21.254:37189 with env {}
W20250809 14:08:27.000336 2440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:27.000487 2440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:27.000517 2440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:27.001976 2440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250809 14:08:27.002019 2440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:27.002041 2440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250809 14:08:27.002060 2440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250809 14:08:27.003551 2440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:43147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.21.254:37189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:37189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.21.254
--webserver_port=34899
--never_fsync=true
--heap_profile_path=/tmp/kudu.2440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:27.003784 2440 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:27.003973 2440 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:27.006145 2445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:27.006263 2440 server_base.cc:1047] running on GCE node
W20250809 14:08:27.006225 2446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:27.006174 2448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:27.006587 2440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:27.006776 2440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:27.007920 2440 hybrid_clock.cc:648] HybridClock initialized: now 1754748507007908 us; error 27 us; skew 500 ppm
I20250809 14:08:27.008986 2440 webserver.cc:489] Webserver started at http://127.31.21.254:34899/ using document root <none> and password file <none>
I20250809 14:08:27.009176 2440 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:27.009213 2440 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:27.010265 2440 fs_manager.cc:714] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20250809 14:08:27.010767 2454 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:27.010895 2440 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:27.010958 2440 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
uuid: "dd5eb26ca1e742f2a81372fea15d8b56"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:27.011178 2440 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:27.019033 2440 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:27.019237 2440 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:27.019328 2440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:27.022655 2440 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.254:37189
I20250809 14:08:27.022692 2506 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.254:37189 every 8 connection(s)
I20250809 14:08:27.022986 2440 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250809 14:08:27.023945 2507 sys_catalog.cc:263] Verifying existing consensus state
I20250809 14:08:27.024463 2507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Bootstrap starting.
I20250809 14:08:27.025776 2507 log.cc:826] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:27.027205 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 2440
I20250809 14:08:27.027451 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.193:40613
--local_ip_for_outbound_sockets=127.31.21.193
--tserver_master_addrs=127.31.21.254:37189
--webserver_port=41677
--webserver_interface=127.31.21.193
--builtin_ntp_servers=127.31.21.212:43147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250809 14:08:27.027899 2507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=18 ignored=0} inserts{seen=13 ignored=0} mutations{seen=10 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:27.028123 2507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Bootstrap complete.
I20250809 14:08:27.029392 2507 raft_consensus.cc:357] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } }
I20250809 14:08:27.029563 2507 raft_consensus.cc:738] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd5eb26ca1e742f2a81372fea15d8b56, State: Initialized, Role: FOLLOWER
I20250809 14:08:27.029676 2507 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 2.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } }
I20250809 14:08:27.029745 2507 raft_consensus.cc:397] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:27.029790 2507 raft_consensus.cc:491] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:27.029834 2507 raft_consensus.cc:3058] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 2 FOLLOWER]: Advancing to term 3
I20250809 14:08:27.030660 2507 raft_consensus.cc:513] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } }
I20250809 14:08:27.030755 2507 leader_election.cc:304] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd5eb26ca1e742f2a81372fea15d8b56; no voters:
I20250809 14:08:27.030877 2507 leader_election.cc:290] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [CANDIDATE]: Term 3 election: Requested vote from peers
I20250809 14:08:27.030927 2511 raft_consensus.cc:2802] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 3 FOLLOWER]: Leader election won for term 3
I20250809 14:08:27.031026 2511 raft_consensus.cc:695] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 3 LEADER]: Becoming Leader. State: Replica: dd5eb26ca1e742f2a81372fea15d8b56, State: Running, Role: LEADER
I20250809 14:08:27.031121 2507 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: configured and running, proceeding with master startup.
I20250809 14:08:27.031180 2511 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 2.18, Last appended by leader: 18, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } }
I20250809 14:08:27.031443 2511 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd5eb26ca1e742f2a81372fea15d8b56. Latest consensus state: current_term: 3 leader_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } } }
I20250809 14:08:27.031473 2512 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } } }
I20250809 14:08:27.031587 2511 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:27.031601 2512 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:27.033039 2527 catalog_manager.cc:1261] Loaded cluster ID: e9f2666a32674795877a734dc9a3e6b0
I20250809 14:08:27.033085 2527 catalog_manager.cc:1554] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: loading cluster ID for follower catalog manager: success
I20250809 14:08:27.034385 2527 catalog_manager.cc:1576] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: acquiring CA information for follower catalog manager: success
I20250809 14:08:27.034687 2527 catalog_manager.cc:1604] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250809 14:08:27.034760 2517 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250809 14:08:27.035202 2517 catalog_manager.cc:671] Loaded metadata for table TestTable2 [id=381ab3452c924286bc13e0224537174f]
I20250809 14:08:27.035276 2517 catalog_manager.cc:671] Loaded metadata for table TestTable1 [id=be1ec9e292124dd7991daaf978d1a366]
I20250809 14:08:27.035328 2517 catalog_manager.cc:671] Loaded metadata for table TestTable [id=c53baba2365e447ab0b9efd060963b37]
I20250809 14:08:27.035459 2517 tablet_loader.cc:96] loaded metadata for tablet 8fe312fd225e401db30ae621e081b279 (table TestTable2 [id=381ab3452c924286bc13e0224537174f])
I20250809 14:08:27.035499 2517 tablet_loader.cc:96] loaded metadata for tablet e035914df64a44b3884a91c9524340d0 (table TestTable [id=c53baba2365e447ab0b9efd060963b37])
I20250809 14:08:27.035523 2517 tablet_loader.cc:96] loaded metadata for tablet f626b930b2c549dfaff430564d98a8fa (table TestTable1 [id=be1ec9e292124dd7991daaf978d1a366])
I20250809 14:08:27.035548 2517 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250809 14:08:27.035604 2517 catalog_manager.cc:1261] Loaded cluster ID: e9f2666a32674795877a734dc9a3e6b0
I20250809 14:08:27.035627 2517 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250809 14:08:27.035775 2517 catalog_manager.cc:1506] Loading token signing keys...
I20250809 14:08:27.035840 2517 catalog_manager.cc:5966] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Loaded TSK: 0
I20250809 14:08:27.035948 2517 catalog_manager.cc:1516] Initializing in-progress tserver states...
W20250809 14:08:27.102944 2510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:27.103089 2510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:27.103108 2510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:27.104453 2510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:27.104502 2510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.193
I20250809 14:08:27.105962 2510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:43147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.193:40613
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.21.193
--webserver_port=41677
--tserver_master_addrs=127.31.21.254:37189
--never_fsync=true
--heap_profile_path=/tmp/kudu.2510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.193
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:27.106153 2510 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:27.106369 2510 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:27.108669 2533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:27.108733 2534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:27.108788 2510 server_base.cc:1047] running on GCE node
W20250809 14:08:27.108826 2536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:27.109051 2510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:27.109247 2510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:27.110383 2510 hybrid_clock.cc:648] HybridClock initialized: now 1754748507110365 us; error 35 us; skew 500 ppm
I20250809 14:08:27.111455 2510 webserver.cc:489] Webserver started at http://127.31.21.193:41677/ using document root <none> and password file <none>
I20250809 14:08:27.111631 2510 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:27.111687 2510 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:27.112819 2510 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:27.113456 2542 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:27.113610 2510 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:27.113677 2510 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "5a52eadefc7043b7bc9cc388cb50cb4d"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:27.113910 2510 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:27.131495 2510 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:27.131718 2510 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:27.131846 2510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:27.132054 2510 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:27.132474 2549 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250809 14:08:27.133381 2510 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250809 14:08:27.133445 2510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:27.133471 2510 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250809 14:08:27.134011 2510 ts_tablet_manager.cc:610] Registered 1 tablets
I20250809 14:08:27.134057 2510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:27.134115 2549 tablet_bootstrap.cc:492] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d: Bootstrap starting.
I20250809 14:08:27.141541 2510 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.193:40613
I20250809 14:08:27.141603 2656 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.193:40613 every 8 connection(s)
I20250809 14:08:27.142042 2510 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250809 14:08:27.144501 2549 log.cc:826] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:27.148368 2657 heartbeater.cc:344] Connected to a master server at 127.31.21.254:37189
I20250809 14:08:27.148459 2657 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:27.148648 2657 heartbeater.cc:507] Master 127.31.21.254:37189 requested a full tablet report, sending...
I20250809 14:08:27.149425 2549 tablet_bootstrap.cc:492] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=600 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:27.149516 2471 ts_manager.cc:194] Registered new tserver with Master: 5a52eadefc7043b7bc9cc388cb50cb4d (127.31.21.193:40613)
I20250809 14:08:27.149677 2549 tablet_bootstrap.cc:492] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d: Bootstrap complete.
I20250809 14:08:27.150478 2549 ts_tablet_manager.cc:1397] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20250809 14:08:27.150542 2471 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.193:34309
I20250809 14:08:27.151223 2549 raft_consensus.cc:357] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 40613 } }
I20250809 14:08:27.151556 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 2510
I20250809 14:08:27.151762 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.194:46267
--local_ip_for_outbound_sockets=127.31.21.194
--tserver_master_addrs=127.31.21.254:37189
--webserver_port=33809
--webserver_interface=127.31.21.194
--builtin_ntp_servers=127.31.21.212:43147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250809 14:08:27.151389 2549 raft_consensus.cc:738] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a52eadefc7043b7bc9cc388cb50cb4d, State: Initialized, Role: FOLLOWER
I20250809 14:08:27.153010 2549 consensus_queue.cc:260] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 1.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 40613 } }
I20250809 14:08:27.153085 2549 raft_consensus.cc:397] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:27.153146 2549 raft_consensus.cc:491] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:27.153195 2549 raft_consensus.cc:3058] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 1 FOLLOWER]: Advancing to term 2
I20250809 14:08:27.153995 2549 raft_consensus.cc:513] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 40613 } }
I20250809 14:08:27.154085 2549 leader_election.cc:304] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a52eadefc7043b7bc9cc388cb50cb4d; no voters:
I20250809 14:08:27.154210 2549 leader_election.cc:290] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [CANDIDATE]: Term 2 election: Requested vote from peers
I20250809 14:08:27.154271 2663 raft_consensus.cc:2802] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 2 FOLLOWER]: Leader election won for term 2
I20250809 14:08:27.154377 2549 ts_tablet_manager.cc:1428] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20250809 14:08:27.154384 2657 heartbeater.cc:499] Master 127.31.21.254:37189 was elected leader, sending a full tablet report...
I20250809 14:08:27.154408 2663 raft_consensus.cc:695] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 2 LEADER]: Becoming Leader. State: Replica: 5a52eadefc7043b7bc9cc388cb50cb4d, State: Running, Role: LEADER
I20250809 14:08:27.154809 2663 consensus_queue.cc:237] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 13, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 40613 } }
I20250809 14:08:27.155347 2471 catalog_manager.cc:5582] T 8fe312fd225e401db30ae621e081b279 P 5a52eadefc7043b7bc9cc388cb50cb4d reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 40613 } health_report { overall_health: HEALTHY } } }
W20250809 14:08:27.227617 2662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:27.227763 2662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:27.227782 2662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:27.229123 2662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:27.229167 2662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.194
I20250809 14:08:27.230561 2662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:43147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.194:46267
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.21.194
--webserver_port=33809
--tserver_master_addrs=127.31.21.254:37189
--never_fsync=true
--heap_profile_path=/tmp/kudu.2662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.194
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:27.230731 2662 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:27.230934 2662 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:27.233366 2675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:27.233383 2673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:27.233384 2672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:27.233556 2662 server_base.cc:1047] running on GCE node
I20250809 14:08:27.233721 2662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:27.233882 2662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:27.234999 2662 hybrid_clock.cc:648] HybridClock initialized: now 1754748507234992 us; error 34 us; skew 500 ppm
I20250809 14:08:27.235965 2662 webserver.cc:489] Webserver started at http://127.31.21.194:33809/ using document root <none> and password file <none>
I20250809 14:08:27.236119 2662 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:27.236169 2662 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:27.237351 2662 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:27.237901 2681 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:27.238035 2662 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:27.238094 2662 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "b7dd44e3fbd744d39cc8a3c3307b3940"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:27.238341 2662 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:27.251091 2662 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:27.251353 2662 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:27.251463 2662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:27.251641 2662 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:27.252017 2688 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250809 14:08:27.252760 2662 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250809 14:08:27.252815 2662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:27.252868 2662 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250809 14:08:27.253368 2662 ts_tablet_manager.cc:610] Registered 1 tablets
I20250809 14:08:27.253399 2662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:27.253463 2688 tablet_bootstrap.cc:492] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: Bootstrap starting.
I20250809 14:08:27.260092 2662 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.194:46267
I20250809 14:08:27.260170 2795 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.194:46267 every 8 connection(s)
I20250809 14:08:27.260435 2662 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250809 14:08:27.260582 2688 log.cc:826] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:27.265278 2796 heartbeater.cc:344] Connected to a master server at 127.31.21.254:37189
I20250809 14:08:27.265364 2796 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:27.265398 2688 tablet_bootstrap.cc:492] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:27.265542 2796 heartbeater.cc:507] Master 127.31.21.254:37189 requested a full tablet report, sending...
I20250809 14:08:27.265674 2688 tablet_bootstrap.cc:492] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: Bootstrap complete.
I20250809 14:08:27.265995 2471 ts_manager.cc:194] Registered new tserver with Master: b7dd44e3fbd744d39cc8a3c3307b3940 (127.31.21.194:46267)
I20250809 14:08:27.266359 2471 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.194:35019
I20250809 14:08:27.266551 2688 ts_tablet_manager.cc:1397] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20250809 14:08:27.267011 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 2662
I20250809 14:08:27.267285 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.195:45239
--local_ip_for_outbound_sockets=127.31.21.195
--tserver_master_addrs=127.31.21.254:37189
--webserver_port=39965
--webserver_interface=127.31.21.195
--builtin_ntp_servers=127.31.21.212:43147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
I20250809 14:08:27.267596 2688 raft_consensus.cc:357] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } }
I20250809 14:08:27.267882 2688 raft_consensus.cc:738] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7dd44e3fbd744d39cc8a3c3307b3940, State: Initialized, Role: FOLLOWER
I20250809 14:08:27.267980 2688 consensus_queue.cc:260] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } }
I20250809 14:08:27.268043 2688 raft_consensus.cc:397] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:27.268093 2688 raft_consensus.cc:491] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:27.268142 2688 raft_consensus.cc:3058] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 1 FOLLOWER]: Advancing to term 2
I20250809 14:08:27.269217 2688 raft_consensus.cc:513] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } }
I20250809 14:08:27.269305 2688 leader_election.cc:304] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7dd44e3fbd744d39cc8a3c3307b3940; no voters:
I20250809 14:08:27.269440 2688 leader_election.cc:290] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [CANDIDATE]: Term 2 election: Requested vote from peers
I20250809 14:08:27.269500 2801 raft_consensus.cc:2802] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 2 FOLLOWER]: Leader election won for term 2
I20250809 14:08:27.269630 2796 heartbeater.cc:499] Master 127.31.21.254:37189 was elected leader, sending a full tablet report...
I20250809 14:08:27.269634 2688 ts_tablet_manager.cc:1428] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20250809 14:08:27.269675 2801 raft_consensus.cc:695] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 2 LEADER]: Becoming Leader. State: Replica: b7dd44e3fbd744d39cc8a3c3307b3940, State: Running, Role: LEADER
I20250809 14:08:27.269815 2801 consensus_queue.cc:237] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } }
I20250809 14:08:27.270316 2471 catalog_manager.cc:5582] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 reported cstate change: term changed from 0 to 2, leader changed from <none> to b7dd44e3fbd744d39cc8a3c3307b3940 (127.31.21.194), VOTER b7dd44e3fbd744d39cc8a3c3307b3940 (127.31.21.194) added. New cstate: current_term: 2 leader_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } health_report { overall_health: HEALTHY } } }
I20250809 14:08:27.275058 2750 consensus_queue.cc:237] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 2.13, Last appended by leader: 12, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } }
I20250809 14:08:27.275419 2801 raft_consensus.cc:2953] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 2 LEADER]: Committing config change with OpId 2.14: config changed from index -1 to 14, NON_VOTER 5a52eadefc7043b7bc9cc388cb50cb4d (127.31.21.193) added. New config: { opid_index: 14 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } } }
I20250809 14:08:27.276445 2458 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet e035914df64a44b3884a91c9524340d0 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20250809 14:08:27.276420 2471 catalog_manager.cc:5582] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 reported cstate change: config changed from index -1 to 14, NON_VOTER 5a52eadefc7043b7bc9cc388cb50cb4d (127.31.21.193) added. New cstate: current_term: 2 leader_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" committed_config { opid_index: 14 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250809 14:08:27.277084 2471 catalog_manager.cc:5260] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet e035914df64a44b3884a91c9524340d0 with cas_config_opid_index 14: no extra replica candidate found for tablet e035914df64a44b3884a91c9524340d0 (table TestTable [id=c53baba2365e447ab0b9efd060963b37]): Not found: could not select location for extra replica: not enough tablet servers to satisfy replica placement policy: the total number of registered tablet servers (2) does not allow for adding an extra replica; consider bringing up more to have at least 4 tablet servers up and running
W20250809 14:08:27.279270 2685 consensus_peers.cc:489] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 -> Peer 5a52eadefc7043b7bc9cc388cb50cb4d (127.31.21.193:40613): Couldn't send request to peer 5a52eadefc7043b7bc9cc388cb50cb4d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e035914df64a44b3884a91c9524340d0. This is attempt 1: this message will repeat every 5th retry.
W20250809 14:08:27.344504 2800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:27.344655 2800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:27.344673 2800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:27.345999 2800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:27.346052 2800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.195
I20250809 14:08:27.347427 2800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:43147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.195:45239
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.21.195
--webserver_port=39965
--tserver_master_addrs=127.31.21.254:37189
--never_fsync=true
--heap_profile_path=/tmp/kudu.2800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.195
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:27.347595 2800 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:27.347764 2800 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:27.350183 2813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:27.350198 2814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:27.350411 2800 server_base.cc:1047] running on GCE node
W20250809 14:08:27.350554 2816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:27.350781 2800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:27.351007 2800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:27.352144 2800 hybrid_clock.cc:648] HybridClock initialized: now 1754748507352125 us; error 25 us; skew 500 ppm
I20250809 14:08:27.353183 2800 webserver.cc:489] Webserver started at http://127.31.21.195:39965/ using document root <none> and password file <none>
I20250809 14:08:27.353354 2800 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:27.353411 2800 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:27.354485 2800 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20250809 14:08:27.355020 2822 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:27.355172 2800 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250809 14:08:27.355247 2800 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "4af13b22c6ad44cf94062caee49dc81b"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:27.355473 2800 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:27.362147 2800 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:27.362325 2800 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:27.362418 2800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:27.362587 2800 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:27.362943 2829 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250809 14:08:27.363660 2800 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250809 14:08:27.363695 2800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:27.363735 2800 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250809 14:08:27.364234 2800 ts_tablet_manager.cc:610] Registered 1 tablets
I20250809 14:08:27.364272 2800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:27.364317 2829 tablet_bootstrap.cc:492] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: Bootstrap starting.
I20250809 14:08:27.369740 2800 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.195:45239
I20250809 14:08:27.369809 2936 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.195:45239 every 8 connection(s)
I20250809 14:08:27.370091 2800 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250809 14:08:27.372079 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 2800
I20250809 14:08:27.372996 2829 log.cc:826] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:27.374634 2937 heartbeater.cc:344] Connected to a master server at 127.31.21.254:37189
I20250809 14:08:27.374729 2937 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:27.374939 2937 heartbeater.cc:507] Master 127.31.21.254:37189 requested a full tablet report, sending...
I20250809 14:08:27.375439 2471 ts_manager.cc:194] Registered new tserver with Master: 4af13b22c6ad44cf94062caee49dc81b (127.31.21.195:45239)
I20250809 14:08:27.375954 2471 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.195:41095
I20250809 14:08:27.377154 2750 consensus_queue.cc:237] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 12, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } } peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: NON_VOTER last_known_addr { host: "127.31.21.195" port: 45239 } attrs { promote: true } }
I20250809 14:08:27.377462 2829 tablet_bootstrap.cc:492] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=600 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:27.377494 2803 raft_consensus.cc:2953] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 [term 2 LEADER]: Committing config change with OpId 2.15: config changed from index 14 to 15, NON_VOTER 4af13b22c6ad44cf94062caee49dc81b (127.31.21.195) added. New config: { opid_index: 15 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } } peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: NON_VOTER last_known_addr { host: "127.31.21.195" port: 45239 } attrs { promote: true } } }
W20250809 14:08:27.377795 2685 consensus_peers.cc:489] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 -> Peer 5a52eadefc7043b7bc9cc388cb50cb4d (127.31.21.193:40613): Couldn't send request to peer 5a52eadefc7043b7bc9cc388cb50cb4d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e035914df64a44b3884a91c9524340d0. This is attempt 1: this message will repeat every 5th retry.
I20250809 14:08:27.377920 2829 tablet_bootstrap.cc:492] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: Bootstrap complete.
I20250809 14:08:27.378669 2458 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet e035914df64a44b3884a91c9524340d0 with cas_config_opid_index 14: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 3)
I20250809 14:08:27.378736 2471 catalog_manager.cc:5582] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 reported cstate change: config changed from index 14 to 15, NON_VOTER 4af13b22c6ad44cf94062caee49dc81b (127.31.21.195) added. New cstate: current_term: 2 leader_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" committed_config { opid_index: 15 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: NON_VOTER last_known_addr { host: "127.31.21.195" port: 45239 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250809 14:08:27.379165 2829 ts_tablet_manager.cc:1397] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.007s
I20250809 14:08:27.380053 2829 raft_consensus.cc:357] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } }
I20250809 14:08:27.380264 2829 raft_consensus.cc:738] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4af13b22c6ad44cf94062caee49dc81b, State: Initialized, Role: FOLLOWER
I20250809 14:08:27.380380 2829 consensus_queue.cc:260] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 1.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } }
I20250809 14:08:27.380434 2829 raft_consensus.cc:397] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:27.380493 2829 raft_consensus.cc:491] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:27.380542 2829 raft_consensus.cc:3058] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 1 FOLLOWER]: Advancing to term 2
I20250809 14:08:27.381466 2829 raft_consensus.cc:513] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } }
I20250809 14:08:27.381508 31831 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250809 14:08:27.381613 2829 leader_election.cc:304] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4af13b22c6ad44cf94062caee49dc81b; no voters:
I20250809 14:08:27.381769 2829 leader_election.cc:290] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [CANDIDATE]: Term 2 election: Requested vote from peers
I20250809 14:08:27.381819 2944 raft_consensus.cc:2802] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 2 FOLLOWER]: Leader election won for term 2
I20250809 14:08:27.381981 2944 raft_consensus.cc:695] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 2 LEADER]: Becoming Leader. State: Replica: 4af13b22c6ad44cf94062caee49dc81b, State: Running, Role: LEADER
I20250809 14:08:27.382004 2829 ts_tablet_manager.cc:1428] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20250809 14:08:27.382143 2944 consensus_queue.cc:237] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 13, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } }
I20250809 14:08:27.382200 31831 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250809 14:08:27.382385 2937 heartbeater.cc:499] Master 127.31.21.254:37189 was elected leader, sending a full tablet report...
I20250809 14:08:27.382632 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 5a52eadefc7043b7bc9cc388cb50cb4d to finish bootstrapping
W20250809 14:08:27.382629 2682 consensus_peers.cc:489] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940 -> Peer 4af13b22c6ad44cf94062caee49dc81b (127.31.21.195:45239): Couldn't send request to peer 4af13b22c6ad44cf94062caee49dc81b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e035914df64a44b3884a91c9524340d0. This is attempt 1: this message will repeat every 5th retry.
I20250809 14:08:27.382778 2471 catalog_manager.cc:5582] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b reported cstate change: term changed from 0 to 2, leader changed from <none> to 4af13b22c6ad44cf94062caee49dc81b (127.31.21.195), VOTER 4af13b22c6ad44cf94062caee49dc81b (127.31.21.195) added. New cstate: current_term: 2 leader_uuid: "4af13b22c6ad44cf94062caee49dc81b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } health_report { overall_health: HEALTHY } } }
I20250809 14:08:27.385814 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver b7dd44e3fbd744d39cc8a3c3307b3940 to finish bootstrapping
I20250809 14:08:27.386039 2891 consensus_queue.cc:237] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 13, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } }
I20250809 14:08:27.386374 2946 raft_consensus.cc:2953] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 2 LEADER]: Committing config change with OpId 2.15: config changed from index -1 to 15, NON_VOTER 5a52eadefc7043b7bc9cc388cb50cb4d (127.31.21.193) added. New config: { opid_index: 15 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } } }
I20250809 14:08:27.387224 2455 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet f626b930b2c549dfaff430564d98a8fa with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20250809 14:08:27.388034 2471 catalog_manager.cc:5582] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b reported cstate change: config changed from index -1 to 15, NON_VOTER 5a52eadefc7043b7bc9cc388cb50cb4d (127.31.21.193) added. New cstate: current_term: 2 leader_uuid: "4af13b22c6ad44cf94062caee49dc81b" committed_config { opid_index: 15 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20250809 14:08:27.388918 2891 consensus_queue.cc:237] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 2.15, Last appended by leader: 13, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } } peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: NON_VOTER last_known_addr { host: "127.31.21.194" port: 46267 } attrs { promote: true } }
I20250809 14:08:27.389144 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 4af13b22c6ad44cf94062caee49dc81b to finish bootstrapping
I20250809 14:08:27.389374 2944 raft_consensus.cc:2953] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b [term 2 LEADER]: Committing config change with OpId 2.16: config changed from index 15 to 16, NON_VOTER b7dd44e3fbd744d39cc8a3c3307b3940 (127.31.21.194) added. New config: { opid_index: 16 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } } peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: NON_VOTER last_known_addr { host: "127.31.21.194" port: 46267 } attrs { promote: true } } }
W20250809 14:08:27.389995 2826 consensus_peers.cc:489] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b -> Peer 5a52eadefc7043b7bc9cc388cb50cb4d (127.31.21.193:40613): Couldn't send request to peer 5a52eadefc7043b7bc9cc388cb50cb4d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f626b930b2c549dfaff430564d98a8fa. This is attempt 1: this message will repeat every 5th retry.
I20250809 14:08:27.390295 2455 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet f626b930b2c549dfaff430564d98a8fa with cas_config_opid_index 15: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20250809 14:08:27.391535 2471 catalog_manager.cc:5582] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b reported cstate change: config changed from index 15 to 16, NON_VOTER b7dd44e3fbd744d39cc8a3c3307b3940 (127.31.21.194) added. New cstate: current_term: 2 leader_uuid: "4af13b22c6ad44cf94062caee49dc81b" committed_config { opid_index: 16 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: NON_VOTER last_known_addr { host: "127.31.21.194" port: 46267 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20250809 14:08:27.391932 2826 consensus_peers.cc:489] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b -> Peer b7dd44e3fbd744d39cc8a3c3307b3940 (127.31.21.194:46267): Couldn't send request to peer b7dd44e3fbd744d39cc8a3c3307b3940. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f626b930b2c549dfaff430564d98a8fa. This is attempt 1: this message will repeat every 5th retry.
I20250809 14:08:27.415055 2871 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250809 14:08:27.415555 2730 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250809 14:08:27.418597 2591 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
dd5eb26ca1e742f2a81372fea15d8b56 | 127.31.21.254:37189 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.31.21.212:43147 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
4af13b22c6ad44cf94062caee49dc81b | 127.31.21.195:45239 | HEALTHY | <none> | 1 | 0
5a52eadefc7043b7bc9cc388cb50cb4d | 127.31.21.193:40613 | HEALTHY | <none> | 1 | 0
b7dd44e3fbd744d39cc8a3c3307b3940 | 127.31.21.194:46267 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.31.21.193 | experimental | 127.31.21.193:40613
local_ip_for_outbound_sockets | 127.31.21.194 | experimental | 127.31.21.194:46267
local_ip_for_outbound_sockets | 127.31.21.195 | experimental | 127.31.21.195:45239
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden | 127.31.21.193:40613
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden | 127.31.21.194:46267
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden | 127.31.21.195:45239
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.31.21.212:43147 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
------------+----+---------+---------------+---------+------------+------------------+-------------
TestTable | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
TestTable1 | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
TestTable2 | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 2
First Quartile | 2
Median | 2
Third Quartile | 3
Maximum | 3
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 3
Tablets | 3
Replicas | 7
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20250809 14:08:27.426088 31831 log_verifier.cc:126] Checking tablet 8fe312fd225e401db30ae621e081b279
I20250809 14:08:27.432473 31831 log_verifier.cc:177] Verified matching terms for 14 ops in tablet 8fe312fd225e401db30ae621e081b279
I20250809 14:08:27.432521 31831 log_verifier.cc:126] Checking tablet e035914df64a44b3884a91c9524340d0
I20250809 14:08:27.437862 31831 log_verifier.cc:177] Verified matching terms for 15 ops in tablet e035914df64a44b3884a91c9524340d0
I20250809 14:08:27.437914 31831 log_verifier.cc:126] Checking tablet f626b930b2c549dfaff430564d98a8fa
I20250809 14:08:27.443338 31831 log_verifier.cc:177] Verified matching terms for 16 ops in tablet f626b930b2c549dfaff430564d98a8fa
I20250809 14:08:27.443436 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 2440
I20250809 14:08:27.449470 31831 minidump.cc:252] Setting minidump size limit to 20M
I20250809 14:08:27.449709 31831 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:27.449836 31831 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:27.451956 2977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:27.451973 2975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:27.451988 2974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:27.452083 31831 server_base.cc:1047] running on GCE node
I20250809 14:08:27.452284 31831 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250809 14:08:27.452325 31831 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250809 14:08:27.452354 31831 hybrid_clock.cc:648] HybridClock initialized: now 1754748507452354 us; error 0 us; skew 500 ppm
I20250809 14:08:27.453043 31831 webserver.cc:489] Webserver started at http://0.0.0.0:42389/ using document root <none> and password file <none>
I20250809 14:08:27.453222 31831 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:27.453259 31831 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:27.453984 31831 fs_manager.cc:714] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:27.454483 2982 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:27.454617 31831 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:27.454682 31831 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
uuid: "dd5eb26ca1e742f2a81372fea15d8b56"
format_stamp: "Formatted at 2025-08-09 14:08:26 on dist-test-slave-mstq"
I20250809 14:08:27.454875 31831 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:27.469204 31831 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:27.469405 31831 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:27.469502 31831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:27.470297 31831 sys_catalog.cc:263] Verifying existing consensus state
W20250809 14:08:27.470603 31831 sys_catalog.cc:243] For a single master config, on-disk Raft master: 127.31.21.254:37189 exists but no master address supplied!
I20250809 14:08:27.471051 31831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Bootstrap starting.
I20250809 14:08:27.477542 31831 log.cc:826] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:27.479825 31831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Bootstrap replayed 1/1 log segments. Stats: ops{read=26 overwritten=0 applied=26 ignored=0} inserts{seen=13 ignored=0} mutations{seen=17 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:27.480065 31831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56: Bootstrap complete.
I20250809 14:08:27.481712 31831 raft_consensus.cc:357] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } }
I20250809 14:08:27.481813 31831 raft_consensus.cc:738] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd5eb26ca1e742f2a81372fea15d8b56, State: Initialized, Role: FOLLOWER
I20250809 14:08:27.481905 31831 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 26, Last appended: 3.26, Last appended by leader: 26, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } }
I20250809 14:08:27.481967 31831 raft_consensus.cc:397] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 3 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:27.482017 31831 raft_consensus.cc:491] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 3 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:27.482064 31831 raft_consensus.cc:3058] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 3 FOLLOWER]: Advancing to term 4
I20250809 14:08:27.482869 31831 raft_consensus.cc:513] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } }
I20250809 14:08:27.482964 31831 leader_election.cc:304] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd5eb26ca1e742f2a81372fea15d8b56; no voters:
I20250809 14:08:27.483115 31831 leader_election.cc:290] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [CANDIDATE]: Term 4 election: Requested vote from peers
I20250809 14:08:27.483178 2989 raft_consensus.cc:2802] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 4 FOLLOWER]: Leader election won for term 4
I20250809 14:08:27.483337 2989 raft_consensus.cc:695] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 4 LEADER]: Becoming Leader. State: Replica: dd5eb26ca1e742f2a81372fea15d8b56, State: Running, Role: LEADER
I20250809 14:08:27.483428 2989 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 26, Committed index: 26, Last appended: 3.26, Last appended by leader: 26, Current term: 4, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } }
I20250809 14:08:27.483678 2990 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 4 leader_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } } }
I20250809 14:08:27.483745 2991 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd5eb26ca1e742f2a81372fea15d8b56. Latest consensus state: current_term: 4 leader_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5eb26ca1e742f2a81372fea15d8b56" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 37189 } } }
I20250809 14:08:27.483841 2990 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:27.483949 2991 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:27.485293 31831 tablet_replica.cc:331] stopping tablet replica
I20250809 14:08:27.485375 31831 raft_consensus.cc:2241] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 4 LEADER]: Raft consensus shutting down.
I20250809 14:08:27.485423 31831 raft_consensus.cc:2270] T 00000000000000000000000000000000 P dd5eb26ca1e742f2a81372fea15d8b56 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250809 14:08:27.485738 31831 master.cc:561] Master@0.0.0.0:7051 shutting down...
W20250809 14:08:27.485837 31831 acceptor_pool.cc:194] Could not shut down acceptor socket on 0.0.0.0:7051: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20250809 14:08:27.496658 31831 master.cc:583] Master@0.0.0.0:7051 shutdown complete.
I20250809 14:08:27.774085 3039 ts_tablet_manager.cc:927] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d: Initiating tablet copy from peer b7dd44e3fbd744d39cc8a3c3307b3940 (127.31.21.194:46267)
I20250809 14:08:27.774286 3039 tablet_copy_client.cc:323] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d: tablet copy: Beginning tablet copy session from remote peer at address 127.31.21.194:46267
I20250809 14:08:27.777963 2770 tablet_copy_service.cc:140] P b7dd44e3fbd744d39cc8a3c3307b3940: Received BeginTabletCopySession request for tablet e035914df64a44b3884a91c9524340d0 from peer 5a52eadefc7043b7bc9cc388cb50cb4d ({username='slave'} at 127.31.21.193:39835)
I20250809 14:08:27.778050 2770 tablet_copy_service.cc:161] P b7dd44e3fbd744d39cc8a3c3307b3940: Beginning new tablet copy session on tablet e035914df64a44b3884a91c9524340d0 from peer 5a52eadefc7043b7bc9cc388cb50cb4d at {username='slave'} at 127.31.21.193:39835: session id = 5a52eadefc7043b7bc9cc388cb50cb4d-e035914df64a44b3884a91c9524340d0
I20250809 14:08:27.778693 2770 tablet_copy_source_session.cc:215] T e035914df64a44b3884a91c9524340d0 P b7dd44e3fbd744d39cc8a3c3307b3940: Tablet Copy: opened 0 blocks and 1 log segments
I20250809 14:08:27.779295 3039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e035914df64a44b3884a91c9524340d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:27.780508 3039 tablet_copy_client.cc:806] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d: tablet copy: Starting download of 0 data blocks...
I20250809 14:08:27.780627 3039 tablet_copy_client.cc:670] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d: tablet copy: Starting download of 1 WAL segments...
I20250809 14:08:27.781409 3039 tablet_copy_client.cc:538] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250809 14:08:27.781942 3039 tablet_bootstrap.cc:492] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d: Bootstrap starting.
I20250809 14:08:27.784699 3039 tablet_bootstrap.cc:492] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:27.785120 3039 tablet_bootstrap.cc:492] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d: Bootstrap complete.
I20250809 14:08:27.786556 3039 ts_tablet_manager.cc:1397] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20250809 14:08:27.786727 3039 raft_consensus.cc:357] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } } peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: NON_VOTER last_known_addr { host: "127.31.21.195" port: 45239 } attrs { promote: true } }
I20250809 14:08:27.786809 3039 raft_consensus.cc:738] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 5a52eadefc7043b7bc9cc388cb50cb4d, State: Initialized, Role: LEARNER
I20250809 14:08:27.786871 3039 consensus_queue.cc:260] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 2.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: true peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 46267 } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } } peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: NON_VOTER last_known_addr { host: "127.31.21.195" port: 45239 } attrs { promote: true } }
I20250809 14:08:27.787187 3039 ts_tablet_manager.cc:1428] T e035914df64a44b3884a91c9524340d0 P 5a52eadefc7043b7bc9cc388cb50cb4d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:27.787425 2770 tablet_copy_service.cc:342] P b7dd44e3fbd744d39cc8a3c3307b3940: Request end of tablet copy session 5a52eadefc7043b7bc9cc388cb50cb4d-e035914df64a44b3884a91c9524340d0 received from {username='slave'} at 127.31.21.193:39835
I20250809 14:08:27.787493 2770 tablet_copy_service.cc:434] P b7dd44e3fbd744d39cc8a3c3307b3940: ending tablet copy session 5a52eadefc7043b7bc9cc388cb50cb4d-e035914df64a44b3884a91c9524340d0 on tablet e035914df64a44b3884a91c9524340d0 with peer 5a52eadefc7043b7bc9cc388cb50cb4d
W20250809 14:08:27.787679 2657 heartbeater.cc:646] Failed to heartbeat to 127.31.21.254:37189 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.21.254:37189: connect: Connection refused (error 111)
I20250809 14:08:27.864042 3039 ts_tablet_manager.cc:927] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d: Initiating tablet copy from peer 4af13b22c6ad44cf94062caee49dc81b (127.31.21.195:45239)
I20250809 14:08:27.864235 3039 tablet_copy_client.cc:323] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d: tablet copy: Beginning tablet copy session from remote peer at address 127.31.21.195:45239
I20250809 14:08:27.867316 2911 tablet_copy_service.cc:140] P 4af13b22c6ad44cf94062caee49dc81b: Received BeginTabletCopySession request for tablet f626b930b2c549dfaff430564d98a8fa from peer 5a52eadefc7043b7bc9cc388cb50cb4d ({username='slave'} at 127.31.21.193:37349)
I20250809 14:08:27.867384 2911 tablet_copy_service.cc:161] P 4af13b22c6ad44cf94062caee49dc81b: Beginning new tablet copy session on tablet f626b930b2c549dfaff430564d98a8fa from peer 5a52eadefc7043b7bc9cc388cb50cb4d at {username='slave'} at 127.31.21.193:37349: session id = 5a52eadefc7043b7bc9cc388cb50cb4d-f626b930b2c549dfaff430564d98a8fa
I20250809 14:08:27.867779 2911 tablet_copy_source_session.cc:215] T f626b930b2c549dfaff430564d98a8fa P 4af13b22c6ad44cf94062caee49dc81b: Tablet Copy: opened 0 blocks and 1 log segments
I20250809 14:08:27.868256 3039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f626b930b2c549dfaff430564d98a8fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:27.869634 3039 tablet_copy_client.cc:806] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d: tablet copy: Starting download of 0 data blocks...
I20250809 14:08:27.869742 3039 tablet_copy_client.cc:670] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d: tablet copy: Starting download of 1 WAL segments...
I20250809 14:08:27.870450 3039 tablet_copy_client.cc:538] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250809 14:08:27.871127 3039 tablet_bootstrap.cc:492] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d: Bootstrap starting.
I20250809 14:08:27.875805 3039 tablet_bootstrap.cc:492] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=600 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:27.876019 3039 tablet_bootstrap.cc:492] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d: Bootstrap complete.
I20250809 14:08:27.876138 3039 ts_tablet_manager.cc:1397] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20250809 14:08:27.876281 3039 raft_consensus.cc:357] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } } peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: NON_VOTER last_known_addr { host: "127.31.21.194" port: 46267 } attrs { promote: true } }
I20250809 14:08:27.876358 3039 raft_consensus.cc:738] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 5a52eadefc7043b7bc9cc388cb50cb4d, State: Initialized, Role: LEARNER
I20250809 14:08:27.876394 3039 consensus_queue.cc:260] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 2.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: true peers { permanent_uuid: "4af13b22c6ad44cf94062caee49dc81b" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 45239 } } peers { permanent_uuid: "5a52eadefc7043b7bc9cc388cb50cb4d" member_type: NON_VOTER last_known_addr { host: "127.31.21.193" port: 40613 } attrs { promote: true } } peers { permanent_uuid: "b7dd44e3fbd744d39cc8a3c3307b3940" member_type: NON_VOTER last_known_addr { host: "127.31.21.194" port: 46267 } attrs { promote: true } }
I20250809 14:08:27.876516 3039 ts_tablet_manager.cc:1428] T f626b930b2c549dfaff430564d98a8fa P 5a52eadefc7043b7bc9cc388cb50cb4d: Time spent starting tablet: real 0.000s user 0.004s sys 0.000s
I20250809 14:08:27.876752 2911 tablet_copy_service.cc:342] P 4af13b22c6ad44cf94062caee49dc81b: Request end of tablet copy session 5a52eadefc7043b7bc9cc388cb50cb4d-f626b930b2c549dfaff430564d98a8fa received from {username='slave'} at 127.31.21.193:37349
I20250809 14:08:27.876816 2911 tablet_copy_service.cc:434] P 4af13b22c6ad44cf94062caee49dc81b: ending tablet copy session 5a52eadefc7043b7bc9cc388cb50cb4d-f626b930b2c549dfaff430564d98a8fa on tablet f626b930b2c549dfaff430564d98a8fa with peer 5a52eadefc7043b7bc9cc388cb50cb4d
/home/jenkins-slave/workspace/build_and_test_flaky/src/kudu/tools/kudu-admin-test.cc:3908: Failure
Value of: stdout2
Expected: has substring "Rebuilt from 3 replicas, of which 0 had errors"
Actual: "Rebuild Report\nTablet Servers\n address | status\n---------------------+--------\n 127.31.21.193:40613 | OK\n 127.31.21.194:46267 | OK\n 127.31.21.195:45239 | OK\nRebuilt from 3 tablet servers, of which 0 had errors\n\nReplicas\n table | tablet | tablet server | status\n-----------+----------------------------------+---------------------+--------\n TestTable | e035914df64a44b3884a91c9524340d0 | 127.31.21.194:46267 | OK\nRebuilt from 1 replicas, of which 0 had errors\n"
I20250809 14:08:27.921053 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 2510
I20250809 14:08:27.925827 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 2662
I20250809 14:08:27.930701 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 2800
2025-08-09T14:08:27Z chronyd exiting
I20250809 14:08:27.945869 31831 test_util.cc:183] -----------------------------------------------
I20250809 14:08:27.945956 31831 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.AdminCliTest.TestRebuildTables.1754748496992630-31831-0
[ FAILED ] AdminCliTest.TestRebuildTables (2031 ms)
[----------] 5 tests from AdminCliTest (10949 ms total)
[----------] 1 test from EnableKudu1097AndDownTS/MoveTabletParamTest
[ RUN ] EnableKudu1097AndDownTS/MoveTabletParamTest.Test/4
I20250809 14:08:27.946363 31831 test_util.cc:276] Using random seed: 964459897
I20250809 14:08:27.946820 31831 ts_itest-base.cc:115] Starting cluster with:
I20250809 14:08:27.946852 31831 ts_itest-base.cc:116] --------------
I20250809 14:08:27.946863 31831 ts_itest-base.cc:117] 5 tablet servers
I20250809 14:08:27.946873 31831 ts_itest-base.cc:118] 3 replicas per TS
I20250809 14:08:27.946887 31831 ts_itest-base.cc:119] --------------
2025-08-09T14:08:27Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-09T14:08:27Z Disabled control of system clock
I20250809 14:08:27.953871 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:34997
--webserver_interface=127.31.21.254
--webserver_port=0
--builtin_ntp_servers=127.31.21.212:45965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.21.254:34997
--raft_prepare_replacement_before_eviction=true with env {}
W20250809 14:08:28.027015 3053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:28.027194 3053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:28.027220 3053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:28.028532 3053 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250809 14:08:28.028570 3053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250809 14:08:28.028592 3053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:28.028607 3053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250809 14:08:28.028625 3053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250809 14:08:28.030001 3053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:45965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.21.254:34997
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:34997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.21.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.3053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:28.030221 3053 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:28.030402 3053 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:28.032534 3059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:28.032541 3058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:28.032660 3061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:28.032714 3053 server_base.cc:1047] running on GCE node
I20250809 14:08:28.032931 3053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:28.033129 3053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:28.034240 3053 hybrid_clock.cc:648] HybridClock initialized: now 1754748508034234 us; error 24 us; skew 500 ppm
I20250809 14:08:28.035272 3053 webserver.cc:489] Webserver started at http://127.31.21.254:33275/ using document root <none> and password file <none>
I20250809 14:08:28.035454 3053 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:28.035493 3053 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:28.035600 3053 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:28.036355 3053 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "c4e3abbaa7ce44ecbe5249897b8714b1"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.036625 3053 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "c4e3abbaa7ce44ecbe5249897b8714b1"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.037748 3053 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250809 14:08:28.038341 3067 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.038493 3053 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:28.038555 3053 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
uuid: "c4e3abbaa7ce44ecbe5249897b8714b1"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.038597 3053 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:28.049024 3053 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:28.049325 3053 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:28.049480 3053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:28.053876 3053 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.254:34997
I20250809 14:08:28.053925 3119 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.254:34997 every 8 connection(s)
I20250809 14:08:28.054286 3053 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250809 14:08:28.054839 3120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:28.056906 3120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1: Bootstrap starting.
I20250809 14:08:28.057456 3120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:28.057579 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 3053
I20250809 14:08:28.057672 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250809 14:08:28.057762 3120 log.cc:826] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:28.058393 3120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1: No bootstrap required, opened a new log
I20250809 14:08:28.059500 3120 raft_consensus.cc:357] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e3abbaa7ce44ecbe5249897b8714b1" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 34997 } }
I20250809 14:08:28.059608 3120 raft_consensus.cc:383] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:28.059640 3120 raft_consensus.cc:738] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4e3abbaa7ce44ecbe5249897b8714b1, State: Initialized, Role: FOLLOWER
I20250809 14:08:28.059756 3120 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e3abbaa7ce44ecbe5249897b8714b1" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 34997 } }
I20250809 14:08:28.059816 3120 raft_consensus.cc:397] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:28.059851 3120 raft_consensus.cc:491] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:28.059899 3120 raft_consensus.cc:3058] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:28.060329 3120 raft_consensus.cc:513] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e3abbaa7ce44ecbe5249897b8714b1" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 34997 } }
I20250809 14:08:28.060420 3120 leader_election.cc:304] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4e3abbaa7ce44ecbe5249897b8714b1; no voters:
I20250809 14:08:28.060535 3120 leader_election.cc:290] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:28.060587 3125 raft_consensus.cc:2802] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:28.060760 3125 raft_consensus.cc:695] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [term 1 LEADER]: Becoming Leader. State: Replica: c4e3abbaa7ce44ecbe5249897b8714b1, State: Running, Role: LEADER
I20250809 14:08:28.060920 3125 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e3abbaa7ce44ecbe5249897b8714b1" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 34997 } }
I20250809 14:08:28.061198 3120 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [sys.catalog]: configured and running, proceeding with master startup.
I20250809 14:08:28.061291 3127 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e3abbaa7ce44ecbe5249897b8714b1. Latest consensus state: current_term: 1 leader_uuid: "c4e3abbaa7ce44ecbe5249897b8714b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e3abbaa7ce44ecbe5249897b8714b1" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 34997 } } }
I20250809 14:08:28.061357 3127 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:28.061520 3126 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4e3abbaa7ce44ecbe5249897b8714b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e3abbaa7ce44ecbe5249897b8714b1" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 34997 } } }
I20250809 14:08:28.061605 3126 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:28.061565 3129 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250809 14:08:28.062105 3129 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250809 14:08:28.063454 3129 catalog_manager.cc:1349] Generated new cluster ID: eb9c408448714aa59338ac64a5aa6b33
I20250809 14:08:28.063505 3129 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250809 14:08:28.103837 3129 catalog_manager.cc:1372] Generated new certificate authority record
I20250809 14:08:28.104344 3129 catalog_manager.cc:1506] Loading token signing keys...
I20250809 14:08:28.109640 3129 catalog_manager.cc:5955] T 00000000000000000000000000000000 P c4e3abbaa7ce44ecbe5249897b8714b1: Generated new TSK 0
I20250809 14:08:28.109810 3129 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250809 14:08:28.113063 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.193:0
--local_ip_for_outbound_sockets=127.31.21.193
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:34997
--builtin_ntp_servers=127.31.21.212:45965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20250809 14:08:28.188573 3144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:28.188740 3144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:28.188776 3144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:28.190061 3144 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250809 14:08:28.190104 3144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:28.190140 3144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.193
I20250809 14:08:28.191468 3144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:45965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:34997
--never_fsync=true
--heap_profile_path=/tmp/kudu.3144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.193
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:28.191656 3144 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:28.191856 3144 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:28.194270 3150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:28.194309 3152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:28.194288 3149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:28.194594 3144 server_base.cc:1047] running on GCE node
I20250809 14:08:28.194756 3144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:28.194958 3144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:28.196126 3144 hybrid_clock.cc:648] HybridClock initialized: now 1754748508196105 us; error 34 us; skew 500 ppm
I20250809 14:08:28.197458 3144 webserver.cc:489] Webserver started at http://127.31.21.193:35987/ using document root <none> and password file <none>
I20250809 14:08:28.197664 3144 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:28.197716 3144 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:28.197827 3144 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:28.198940 3144 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "12d5143c026941589c4a811dda174be7"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.199311 3144 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "12d5143c026941589c4a811dda174be7"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.200613 3144 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20250809 14:08:28.201453 3158 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.201620 3144 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.201679 3144 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "12d5143c026941589c4a811dda174be7"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.201740 3144 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:28.229501 3144 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:28.229710 3144 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:28.229815 3144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:28.229993 3144 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:28.230258 3144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:28.230288 3144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.230315 3144 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:28.230340 3144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.235411 3144 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.193:44321
I20250809 14:08:28.235505 3271 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.193:44321 every 8 connection(s)
I20250809 14:08:28.235769 3144 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250809 14:08:28.237380 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 3144
I20250809 14:08:28.237509 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250809 14:08:28.238619 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.194:0
--local_ip_for_outbound_sockets=127.31.21.194
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:34997
--builtin_ntp_servers=127.31.21.212:45965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20250809 14:08:28.240569 3272 heartbeater.cc:344] Connected to a master server at 127.31.21.254:34997
I20250809 14:08:28.240655 3272 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:28.240834 3272 heartbeater.cc:507] Master 127.31.21.254:34997 requested a full tablet report, sending...
I20250809 14:08:28.241276 3084 ts_manager.cc:194] Registered new tserver with Master: 12d5143c026941589c4a811dda174be7 (127.31.21.193:44321)
I20250809 14:08:28.241881 3084 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.193:39513
W20250809 14:08:28.311937 3275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:28.312104 3275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:28.312153 3275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:28.313544 3275 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250809 14:08:28.313585 3275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:28.313621 3275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.194
I20250809 14:08:28.315021 3275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:45965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:34997
--never_fsync=true
--heap_profile_path=/tmp/kudu.3275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.194
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:28.315246 3275 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:28.315454 3275 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:28.317799 3283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:28.317855 3275 server_base.cc:1047] running on GCE node
W20250809 14:08:28.317826 3281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:28.317826 3280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:28.318161 3275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:28.318352 3275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:28.319471 3275 hybrid_clock.cc:648] HybridClock initialized: now 1754748508319461 us; error 29 us; skew 500 ppm
I20250809 14:08:28.320454 3275 webserver.cc:489] Webserver started at http://127.31.21.194:43343/ using document root <none> and password file <none>
I20250809 14:08:28.320619 3275 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:28.320650 3275 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:28.320726 3275 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:28.321484 3275 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "a80b3ac730854f49828b3390456e9505"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.321720 3275 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "a80b3ac730854f49828b3390456e9505"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.322752 3275 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250809 14:08:28.323336 3289 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.323480 3275 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250809 14:08:28.323544 3275 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "a80b3ac730854f49828b3390456e9505"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.323586 3275 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:28.342741 3275 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:28.342953 3275 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:28.343034 3275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:28.343209 3275 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:28.343506 3275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:28.343537 3275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.343571 3275 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:28.343596 3275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.348932 3275 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.194:34729
I20250809 14:08:28.349018 3402 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.194:34729 every 8 connection(s)
I20250809 14:08:28.349427 3275 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20250809 14:08:28.353435 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 3275
I20250809 14:08:28.353451 3403 heartbeater.cc:344] Connected to a master server at 127.31.21.254:34997
I20250809 14:08:28.353543 3403 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:28.353547 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20250809 14:08:28.353763 3403 heartbeater.cc:507] Master 127.31.21.254:34997 requested a full tablet report, sending...
I20250809 14:08:28.354113 3083 ts_manager.cc:194] Registered new tserver with Master: a80b3ac730854f49828b3390456e9505 (127.31.21.194:34729)
I20250809 14:08:28.354528 3083 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.194:47041
I20250809 14:08:28.354578 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.195:0
--local_ip_for_outbound_sockets=127.31.21.195
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:34997
--builtin_ntp_servers=127.31.21.212:45965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
W20250809 14:08:28.426076 3406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:28.426234 3406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:28.426254 3406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:28.427628 3406 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250809 14:08:28.427668 3406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:28.427690 3406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.195
I20250809 14:08:28.429066 3406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:45965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:34997
--never_fsync=true
--heap_profile_path=/tmp/kudu.3406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.195
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:28.429258 3406 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:28.429422 3406 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:28.431689 3411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:28.431792 3406 server_base.cc:1047] running on GCE node
W20250809 14:08:28.431666 3412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:28.431684 3414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:28.431998 3406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:28.432204 3406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:28.433341 3406 hybrid_clock.cc:648] HybridClock initialized: now 1754748508433326 us; error 27 us; skew 500 ppm
I20250809 14:08:28.434386 3406 webserver.cc:489] Webserver started at http://127.31.21.195:35237/ using document root <none> and password file <none>
I20250809 14:08:28.434572 3406 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:28.434619 3406 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:28.434741 3406 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:28.435536 3406 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "bbf469c9d43b42ba9ea69bb44baa07c3"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.435822 3406 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "bbf469c9d43b42ba9ea69bb44baa07c3"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.436926 3406 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250809 14:08:28.437534 3420 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.437675 3406 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:28.437741 3406 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "bbf469c9d43b42ba9ea69bb44baa07c3"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.437795 3406 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:28.447786 3406 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:28.447986 3406 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:28.448086 3406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:28.448289 3406 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:28.448572 3406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:28.448619 3406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.448647 3406 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:28.448666 3406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.453860 3406 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.195:43783
I20250809 14:08:28.453926 3533 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.195:43783 every 8 connection(s)
I20250809 14:08:28.454198 3406 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20250809 14:08:28.458817 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 3406
I20250809 14:08:28.458909 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20250809 14:08:28.459968 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.196:0
--local_ip_for_outbound_sockets=127.31.21.196
--webserver_interface=127.31.21.196
--webserver_port=0
--tserver_master_addrs=127.31.21.254:34997
--builtin_ntp_servers=127.31.21.212:45965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20250809 14:08:28.491984 3534 heartbeater.cc:344] Connected to a master server at 127.31.21.254:34997
I20250809 14:08:28.492126 3534 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:28.492378 3534 heartbeater.cc:507] Master 127.31.21.254:34997 requested a full tablet report, sending...
I20250809 14:08:28.492743 3083 ts_manager.cc:194] Registered new tserver with Master: bbf469c9d43b42ba9ea69bb44baa07c3 (127.31.21.195:43783)
I20250809 14:08:28.493311 3083 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.195:46075
W20250809 14:08:28.534602 3537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:28.534758 3537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:28.534775 3537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:28.535956 3537 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250809 14:08:28.535989 3537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:28.536007 3537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.196
I20250809 14:08:28.537355 3537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:45965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.31.21.196
--webserver_port=0
--tserver_master_addrs=127.31.21.254:34997
--never_fsync=true
--heap_profile_path=/tmp/kudu.3537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.196
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:28.537513 3537 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:28.537696 3537 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:28.540096 3545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:28.540187 3537 server_base.cc:1047] running on GCE node
W20250809 14:08:28.540098 3543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:28.540264 3542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:28.540446 3537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:28.540633 3537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:28.541754 3537 hybrid_clock.cc:648] HybridClock initialized: now 1754748508541741 us; error 25 us; skew 500 ppm
I20250809 14:08:28.542742 3537 webserver.cc:489] Webserver started at http://127.31.21.196:39145/ using document root <none> and password file <none>
I20250809 14:08:28.542920 3537 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:28.542964 3537 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:28.543074 3537 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:28.543852 3537 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "7f6ea904d7504557839307891ec6fbdd"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.544152 3537 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "7f6ea904d7504557839307891ec6fbdd"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.545241 3537 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250809 14:08:28.545836 3551 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.545984 3537 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250809 14:08:28.546049 3537 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "7f6ea904d7504557839307891ec6fbdd"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.546103 3537 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:28.554675 3537 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:28.554857 3537 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:28.554960 3537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:28.555148 3537 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:28.555408 3537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:28.555438 3537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.555471 3537 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:28.555501 3537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.560693 3537 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.196:33941
I20250809 14:08:28.560745 3664 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.196:33941 every 8 connection(s)
I20250809 14:08:28.561079 3537 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20250809 14:08:28.563995 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 3537
I20250809 14:08:28.564086 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20250809 14:08:28.565191 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.197:0
--local_ip_for_outbound_sockets=127.31.21.197
--webserver_interface=127.31.21.197
--webserver_port=0
--tserver_master_addrs=127.31.21.254:34997
--builtin_ntp_servers=127.31.21.212:45965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=true with env {}
I20250809 14:08:28.565397 3665 heartbeater.cc:344] Connected to a master server at 127.31.21.254:34997
I20250809 14:08:28.565483 3665 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:28.565686 3665 heartbeater.cc:507] Master 127.31.21.254:34997 requested a full tablet report, sending...
I20250809 14:08:28.566002 3083 ts_manager.cc:194] Registered new tserver with Master: 7f6ea904d7504557839307891ec6fbdd (127.31.21.196:33941)
I20250809 14:08:28.566354 3083 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.196:57825
W20250809 14:08:28.637916 3668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:28.638065 3668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:28.638083 3668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:28.639361 3668 flags.cc:425] Enabled experimental flag: --raft_prepare_replacement_before_eviction=true
W20250809 14:08:28.639400 3668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:28.639420 3668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.197
I20250809 14:08:28.640904 3668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:45965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.197:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.31.21.197
--webserver_port=0
--tserver_master_addrs=127.31.21.254:34997
--never_fsync=true
--heap_profile_path=/tmp/kudu.3668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.197
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:28.641117 3668 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:28.641330 3668 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:28.643729 3673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:28.643829 3668 server_base.cc:1047] running on GCE node
W20250809 14:08:28.643729 3676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:28.643915 3674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:28.644084 3668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:28.644274 3668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:28.645398 3668 hybrid_clock.cc:648] HybridClock initialized: now 1754748508645388 us; error 24 us; skew 500 ppm
I20250809 14:08:28.646406 3668 webserver.cc:489] Webserver started at http://127.31.21.197:42439/ using document root <none> and password file <none>
I20250809 14:08:28.646582 3668 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:28.646625 3668 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:28.646744 3668 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:28.647569 3668 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "7d3027312e8d49d384a6ebdfe7235163"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.647846 3668 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "7d3027312e8d49d384a6ebdfe7235163"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.648967 3668 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250809 14:08:28.649698 3682 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.649855 3668 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:28.649921 3668 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "7d3027312e8d49d384a6ebdfe7235163"
format_stamp: "Formatted at 2025-08-09 14:08:28 on dist-test-slave-mstq"
I20250809 14:08:28.649981 3668 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:28.664641 3668 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:28.664927 3668 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:28.665041 3668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:28.665247 3668 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:28.665532 3668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:28.665565 3668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.665601 3668 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:28.665629 3668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:28.670871 3668 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.197:40821
I20250809 14:08:28.670938 3795 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.197:40821 every 8 connection(s)
I20250809 14:08:28.671219 3668 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20250809 14:08:28.675222 3796 heartbeater.cc:344] Connected to a master server at 127.31.21.254:34997
I20250809 14:08:28.675308 3796 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:28.675484 3796 heartbeater.cc:507] Master 127.31.21.254:34997 requested a full tablet report, sending...
I20250809 14:08:28.675799 3083 ts_manager.cc:194] Registered new tserver with Master: 7d3027312e8d49d384a6ebdfe7235163 (127.31.21.197:40821)
I20250809 14:08:28.676167 3083 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.197:34839
I20250809 14:08:28.679538 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 3668
I20250809 14:08:28.679623 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.EnableKudu1097AndDownTS_MoveTabletParamTest.Test_4.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20250809 14:08:28.680653 31831 external_mini_cluster.cc:949] 5 TS(s) registered with all masters
I20250809 14:08:28.686033 3083 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53860:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20250809 14:08:28.693316 3468 tablet_service.cc:1468] Processing CreateTablet for tablet 97fa51b1189f43b083af5682dc653d7e (DEFAULT_TABLE table=TestTable [id=3bc01411a7f841e3a8b0694a31bd0668]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:28.693625 3468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97fa51b1189f43b083af5682dc653d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:28.693560 3337 tablet_service.cc:1468] Processing CreateTablet for tablet 97fa51b1189f43b083af5682dc653d7e (DEFAULT_TABLE table=TestTable [id=3bc01411a7f841e3a8b0694a31bd0668]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:28.693687 3730 tablet_service.cc:1468] Processing CreateTablet for tablet 97fa51b1189f43b083af5682dc653d7e (DEFAULT_TABLE table=TestTable [id=3bc01411a7f841e3a8b0694a31bd0668]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:28.693782 3337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97fa51b1189f43b083af5682dc653d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:28.693950 3730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97fa51b1189f43b083af5682dc653d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:28.695786 3813 tablet_bootstrap.cc:492] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163: Bootstrap starting.
I20250809 14:08:28.696154 3815 tablet_bootstrap.cc:492] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505: Bootstrap starting.
I20250809 14:08:28.696740 3813 tablet_bootstrap.cc:654] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:28.696740 3815 tablet_bootstrap.cc:654] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:28.697059 3815 log.cc:826] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:28.697059 3813 log.cc:826] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:28.697352 3814 tablet_bootstrap.cc:492] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3: Bootstrap starting.
I20250809 14:08:28.697696 3815 tablet_bootstrap.cc:492] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505: No bootstrap required, opened a new log
I20250809 14:08:28.697753 3813 tablet_bootstrap.cc:492] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163: No bootstrap required, opened a new log
I20250809 14:08:28.697762 3815 ts_tablet_manager.cc:1397] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250809 14:08:28.697854 3813 ts_tablet_manager.cc:1397] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250809 14:08:28.697929 3814 tablet_bootstrap.cc:654] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:28.698210 3814 log.cc:826] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:28.698801 3814 tablet_bootstrap.cc:492] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3: No bootstrap required, opened a new log
I20250809 14:08:28.698865 3814 ts_tablet_manager.cc:1397] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:28.699105 3813 raft_consensus.cc:357] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3027312e8d49d384a6ebdfe7235163" member_type: VOTER last_known_addr { host: "127.31.21.197" port: 40821 } } peers { permanent_uuid: "bbf469c9d43b42ba9ea69bb44baa07c3" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 43783 } } peers { permanent_uuid: "a80b3ac730854f49828b3390456e9505" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34729 } }
I20250809 14:08:28.699285 3813 raft_consensus.cc:383] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:28.699339 3813 raft_consensus.cc:738] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d3027312e8d49d384a6ebdfe7235163, State: Initialized, Role: FOLLOWER
I20250809 14:08:28.699285 3815 raft_consensus.cc:357] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3027312e8d49d384a6ebdfe7235163" member_type: VOTER last_known_addr { host: "127.31.21.197" port: 40821 } } peers { permanent_uuid: "bbf469c9d43b42ba9ea69bb44baa07c3" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 43783 } } peers { permanent_uuid: "a80b3ac730854f49828b3390456e9505" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34729 } }
I20250809 14:08:28.699450 3815 raft_consensus.cc:383] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:28.699436 3813 consensus_queue.cc:260] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3027312e8d49d384a6ebdfe7235163" member_type: VOTER last_known_addr { host: "127.31.21.197" port: 40821 } } peers { permanent_uuid: "bbf469c9d43b42ba9ea69bb44baa07c3" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 43783 } } peers { permanent_uuid: "a80b3ac730854f49828b3390456e9505" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34729 } }
I20250809 14:08:28.699496 3815 raft_consensus.cc:738] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a80b3ac730854f49828b3390456e9505, State: Initialized, Role: FOLLOWER
I20250809 14:08:28.699591 3815 consensus_queue.cc:260] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3027312e8d49d384a6ebdfe7235163" member_type: VOTER last_known_addr { host: "127.31.21.197" port: 40821 } } peers { permanent_uuid: "bbf469c9d43b42ba9ea69bb44baa07c3" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 43783 } } peers { permanent_uuid: "a80b3ac730854f49828b3390456e9505" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34729 } }
I20250809 14:08:28.699653 3813 ts_tablet_manager.cc:1428] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250809 14:08:28.699786 3796 heartbeater.cc:499] Master 127.31.21.254:34997 was elected leader, sending a full tablet report...
I20250809 14:08:28.699819 3815 ts_tablet_manager.cc:1428] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250809 14:08:28.699857 3403 heartbeater.cc:499] Master 127.31.21.254:34997 was elected leader, sending a full tablet report...
I20250809 14:08:28.700634 3814 raft_consensus.cc:357] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3027312e8d49d384a6ebdfe7235163" member_type: VOTER last_known_addr { host: "127.31.21.197" port: 40821 } } peers { permanent_uuid: "bbf469c9d43b42ba9ea69bb44baa07c3" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 43783 } } peers { permanent_uuid: "a80b3ac730854f49828b3390456e9505" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34729 } }
I20250809 14:08:28.700757 3814 raft_consensus.cc:383] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:28.700779 3814 raft_consensus.cc:738] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbf469c9d43b42ba9ea69bb44baa07c3, State: Initialized, Role: FOLLOWER
I20250809 14:08:28.700889 3814 consensus_queue.cc:260] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3027312e8d49d384a6ebdfe7235163" member_type: VOTER last_known_addr { host: "127.31.21.197" port: 40821 } } peers { permanent_uuid: "bbf469c9d43b42ba9ea69bb44baa07c3" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 43783 } } peers { permanent_uuid: "a80b3ac730854f49828b3390456e9505" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34729 } }
I20250809 14:08:28.701093 3814 ts_tablet_manager.cc:1428] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:28.701143 3534 heartbeater.cc:499] Master 127.31.21.254:34997 was elected leader, sending a full tablet report...
W20250809 14:08:28.704587 3535 tablet.cc:2378] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250809 14:08:28.781029 3820 raft_consensus.cc:491] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250809 14:08:28.781164 3820 raft_consensus.cc:513] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3027312e8d49d384a6ebdfe7235163" member_type: VOTER last_known_addr { host: "127.31.21.197" port: 40821 } } peers { permanent_uuid: "bbf469c9d43b42ba9ea69bb44baa07c3" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 43783 } } peers { permanent_uuid: "a80b3ac730854f49828b3390456e9505" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34729 } }
I20250809 14:08:28.781512 3820 leader_election.cc:290] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d3027312e8d49d384a6ebdfe7235163 (127.31.21.197:40821), bbf469c9d43b42ba9ea69bb44baa07c3 (127.31.21.195:43783)
I20250809 14:08:28.784337 3750 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "97fa51b1189f43b083af5682dc653d7e" candidate_uuid: "a80b3ac730854f49828b3390456e9505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d3027312e8d49d384a6ebdfe7235163" is_pre_election: true
I20250809 14:08:28.784461 3750 raft_consensus.cc:2466] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a80b3ac730854f49828b3390456e9505 in term 0.
I20250809 14:08:28.784677 3293 leader_election.cc:304] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d3027312e8d49d384a6ebdfe7235163, a80b3ac730854f49828b3390456e9505; no voters:
I20250809 14:08:28.784775 3488 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "97fa51b1189f43b083af5682dc653d7e" candidate_uuid: "a80b3ac730854f49828b3390456e9505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbf469c9d43b42ba9ea69bb44baa07c3" is_pre_election: true
I20250809 14:08:28.784821 3820 raft_consensus.cc:2802] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250809 14:08:28.784899 3820 raft_consensus.cc:491] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250809 14:08:28.784945 3820 raft_consensus.cc:3058] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:28.784953 3488 raft_consensus.cc:2466] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a80b3ac730854f49828b3390456e9505 in term 0.
I20250809 14:08:28.785630 3820 raft_consensus.cc:513] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3027312e8d49d384a6ebdfe7235163" member_type: VOTER last_known_addr { host: "127.31.21.197" port: 40821 } } peers { permanent_uuid: "bbf469c9d43b42ba9ea69bb44baa07c3" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 43783 } } peers { permanent_uuid: "a80b3ac730854f49828b3390456e9505" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34729 } }
I20250809 14:08:28.785750 3820 leader_election.cc:290] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [CANDIDATE]: Term 1 election: Requested vote from peers 7d3027312e8d49d384a6ebdfe7235163 (127.31.21.197:40821), bbf469c9d43b42ba9ea69bb44baa07c3 (127.31.21.195:43783)
I20250809 14:08:28.785919 3750 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "97fa51b1189f43b083af5682dc653d7e" candidate_uuid: "a80b3ac730854f49828b3390456e9505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d3027312e8d49d384a6ebdfe7235163"
I20250809 14:08:28.785986 3750 raft_consensus.cc:3058] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:28.786528 3750 raft_consensus.cc:2466] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a80b3ac730854f49828b3390456e9505 in term 1.
I20250809 14:08:28.786612 3488 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "97fa51b1189f43b083af5682dc653d7e" candidate_uuid: "a80b3ac730854f49828b3390456e9505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbf469c9d43b42ba9ea69bb44baa07c3"
I20250809 14:08:28.786686 3293 leader_election.cc:304] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d3027312e8d49d384a6ebdfe7235163, a80b3ac730854f49828b3390456e9505; no voters:
I20250809 14:08:28.786686 3488 raft_consensus.cc:3058] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:28.786759 3820 raft_consensus.cc:2802] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:28.786895 3820 raft_consensus.cc:695] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [term 1 LEADER]: Becoming Leader. State: Replica: a80b3ac730854f49828b3390456e9505, State: Running, Role: LEADER
I20250809 14:08:28.786986 3820 consensus_queue.cc:237] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3027312e8d49d384a6ebdfe7235163" member_type: VOTER last_known_addr { host: "127.31.21.197" port: 40821 } } peers { permanent_uuid: "bbf469c9d43b42ba9ea69bb44baa07c3" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 43783 } } peers { permanent_uuid: "a80b3ac730854f49828b3390456e9505" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34729 } }
I20250809 14:08:28.787457 3488 raft_consensus.cc:2466] T 97fa51b1189f43b083af5682dc653d7e P bbf469c9d43b42ba9ea69bb44baa07c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a80b3ac730854f49828b3390456e9505 in term 1.
I20250809 14:08:28.787670 3084 catalog_manager.cc:5582] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 reported cstate change: term changed from 0 to 1, leader changed from <none> to a80b3ac730854f49828b3390456e9505 (127.31.21.194). New cstate: current_term: 1 leader_uuid: "a80b3ac730854f49828b3390456e9505" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d3027312e8d49d384a6ebdfe7235163" member_type: VOTER last_known_addr { host: "127.31.21.197" port: 40821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbf469c9d43b42ba9ea69bb44baa07c3" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 43783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a80b3ac730854f49828b3390456e9505" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 34729 } health_report { overall_health: HEALTHY } } }
I20250809 14:08:28.807096 31831 external_mini_cluster.cc:949] 5 TS(s) registered with all masters
I20250809 14:08:28.807564 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver a80b3ac730854f49828b3390456e9505 to finish bootstrapping
I20250809 14:08:28.810815 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver bbf469c9d43b42ba9ea69bb44baa07c3 to finish bootstrapping
I20250809 14:08:28.813848 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 7d3027312e8d49d384a6ebdfe7235163 to finish bootstrapping
I20250809 14:08:28.816596 31831 test_util.cc:276] Using random seed: 965330131
I20250809 14:08:28.821370 31831 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20250809 14:08:28.821508 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 3406
I20250809 14:08:28.826530 3750 raft_consensus.cc:1273] T 97fa51b1189f43b083af5682dc653d7e P 7d3027312e8d49d384a6ebdfe7235163 [term 1 FOLLOWER]: Refusing update from remote peer a80b3ac730854f49828b3390456e9505: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250809 14:08:28.826820 3820 consensus_queue.cc:1035] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d3027312e8d49d384a6ebdfe7235163" member_type: VOTER last_known_addr { host: "127.31.21.197" port: 40821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250809 14:08:28.828409 3291 connection.cc:537] client connection to 127.31.21.195:43783 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250809 14:08:28.828508 3291 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250809 14:08:28.829437 3822 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 14:08:28.828832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.21.195:43783 (local address 127.31.21.194:36037)
0809 14:08:28.828904 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 14:08:28.828914 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 14:08:28.828952 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 14:08:28.829318 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.21.195:43783: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":55}
I20250809 14:08:28.829947 3834 mvcc.cc:204] Tried to move back new op lower bound from 7187449892151205888 to 7187449891992043520. Current Snapshot: MvccSnapshot[applied={T|T < 7187449892151205888 or (T in {7187449892151205888})}]
W20250809 14:08:28.829965 3291 consensus_peers.cc:489] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 -> Peer bbf469c9d43b42ba9ea69bb44baa07c3 (127.31.21.195:43783): Couldn't send request to peer bbf469c9d43b42ba9ea69bb44baa07c3. Status: Network error: Client connection negotiation failed: client connection to 127.31.21.195:43783: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20250809 14:08:28.981292 3730 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:28.985275 3337 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250809 14:08:28.985409 3598 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:28.985457 3206 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:29.242655 3272 heartbeater.cc:499] Master 127.31.21.254:34997 was elected leader, sending a full tablet report...
I20250809 14:08:29.567137 3665 heartbeater.cc:499] Master 127.31.21.254:34997 was elected leader, sending a full tablet report...
I20250809 14:08:30.029227 3337 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250809 14:08:30.030226 3206 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:30.031473 3598 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:30.040565 3730 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:31.073280 3206 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:31.073485 3337 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250809 14:08:31.075719 3598 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:31.077234 3730 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250809 14:08:31.284251 3291 consensus_peers.cc:489] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 -> Peer bbf469c9d43b42ba9ea69bb44baa07c3 (127.31.21.195:43783): Couldn't send request to peer bbf469c9d43b42ba9ea69bb44baa07c3. Status: Network error: Client connection negotiation failed: client connection to 127.31.21.195:43783: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20250809 14:08:32.118650 3206 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:32.118623 3730 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:32.123566 3598 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:32.129972 3337 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250809 14:08:33.168962 3598 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:33.172541 3730 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:33.174391 3337 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250809 14:08:33.177873 3206 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:33.751242 3983 consensus_queue.cc:786] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 [LEADER]: Peer bbf469c9d43b42ba9ea69bb44baa07c3 is lagging by at least 342 ops behind the committed index
W20250809 14:08:33.755214 3291 consensus_peers.cc:489] T 97fa51b1189f43b083af5682dc653d7e P a80b3ac730854f49828b3390456e9505 -> Peer bbf469c9d43b42ba9ea69bb44baa07c3 (127.31.21.195:43783): Couldn't send request to peer bbf469c9d43b42ba9ea69bb44baa07c3. Status: Network error: Client connection negotiation failed: client connection to 127.31.21.195:43783: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20250809 14:08:34.366288 3291 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.21.195:43783: connect: Connection refused (error 111) [suppressed 10 similar messages]
I20250809 14:08:34.377045 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 3144
I20250809 14:08:34.381893 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 3275
I20250809 14:08:34.386924 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 3537
I20250809 14:08:34.391243 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 3668
I20250809 14:08:34.395795 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 3053
2025-08-09T14:08:34Z chronyd exiting
[ OK ] EnableKudu1097AndDownTS/MoveTabletParamTest.Test/4 (6470 ms)
[----------] 1 test from EnableKudu1097AndDownTS/MoveTabletParamTest (6470 ms total)
[----------] 1 test from ListTableCliSimpleParamTest
[ RUN ] ListTableCliSimpleParamTest.TestListTables/2
I20250809 14:08:34.417403 31831 test_util.cc:276] Using random seed: 970930935
I20250809 14:08:34.417893 31831 ts_itest-base.cc:115] Starting cluster with:
I20250809 14:08:34.417943 31831 ts_itest-base.cc:116] --------------
I20250809 14:08:34.417955 31831 ts_itest-base.cc:117] 1 tablet servers
I20250809 14:08:34.417965 31831 ts_itest-base.cc:118] 1 replicas per TS
I20250809 14:08:34.417975 31831 ts_itest-base.cc:119] --------------
2025-08-09T14:08:34Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-09T14:08:34Z Disabled control of system clock
I20250809 14:08:34.424980 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:46271
--webserver_interface=127.31.21.254
--webserver_port=0
--builtin_ntp_servers=127.31.21.212:37435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.21.254:46271 with env {}
W20250809 14:08:34.501523 4042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:34.501693 4042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:34.501710 4042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:34.503186 4042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250809 14:08:34.503234 4042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:34.503249 4042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250809 14:08:34.503261 4042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250809 14:08:34.504778 4042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:37435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.21.254:46271
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:46271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.31.21.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.4042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:34.505048 4042 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:34.505268 4042 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250809 14:08:34.507876 4042 server_base.cc:1047] running on GCE node
W20250809 14:08:34.507830 4047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:34.507851 4048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:34.507956 4050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:34.508351 4042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:34.508586 4042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:34.509742 4042 hybrid_clock.cc:648] HybridClock initialized: now 1754748514509717 us; error 35 us; skew 500 ppm
I20250809 14:08:34.510922 4042 webserver.cc:489] Webserver started at http://127.31.21.254:34427/ using document root <none> and password file <none>
I20250809 14:08:34.511125 4042 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:34.511166 4042 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:34.511279 4042 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:34.512159 4042 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "cdcde223499846178c470a88a31e0eba"
format_stamp: "Formatted at 2025-08-09 14:08:34 on dist-test-slave-mstq"
I20250809 14:08:34.512461 4042 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "cdcde223499846178c470a88a31e0eba"
format_stamp: "Formatted at 2025-08-09 14:08:34 on dist-test-slave-mstq"
I20250809 14:08:34.513691 4042 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20250809 14:08:34.514508 4056 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:34.514706 4042 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20250809 14:08:34.514797 4042 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
uuid: "cdcde223499846178c470a88a31e0eba"
format_stamp: "Formatted at 2025-08-09 14:08:34 on dist-test-slave-mstq"
I20250809 14:08:34.514870 4042 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:34.529011 4042 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:34.529273 4042 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:34.529358 4042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:34.532871 4042 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.254:46271
I20250809 14:08:34.532948 4108 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.254:46271 every 8 connection(s)
I20250809 14:08:34.533252 4042 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20250809 14:08:34.533846 4109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:34.536013 4109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba: Bootstrap starting.
I20250809 14:08:34.536624 4109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:34.536923 4109 log.cc:826] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:34.537520 4109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba: No bootstrap required, opened a new log
I20250809 14:08:34.538667 4109 raft_consensus.cc:357] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdcde223499846178c470a88a31e0eba" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 46271 } }
I20250809 14:08:34.538774 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 4042
I20250809 14:08:34.538787 4109 raft_consensus.cc:383] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:34.538825 4109 raft_consensus.cc:738] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdcde223499846178c470a88a31e0eba, State: Initialized, Role: FOLLOWER
I20250809 14:08:34.538863 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/master-0/wal/instance
I20250809 14:08:34.538957 4109 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdcde223499846178c470a88a31e0eba" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 46271 } }
I20250809 14:08:34.539036 4109 raft_consensus.cc:397] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:34.539083 4109 raft_consensus.cc:491] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:34.539134 4109 raft_consensus.cc:3058] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:34.539887 4109 raft_consensus.cc:513] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdcde223499846178c470a88a31e0eba" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 46271 } }
I20250809 14:08:34.539994 4109 leader_election.cc:304] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdcde223499846178c470a88a31e0eba; no voters:
I20250809 14:08:34.540149 4109 leader_election.cc:290] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:34.540192 4114 raft_consensus.cc:2802] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:34.540297 4114 raft_consensus.cc:695] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [term 1 LEADER]: Becoming Leader. State: Replica: cdcde223499846178c470a88a31e0eba, State: Running, Role: LEADER
I20250809 14:08:34.540392 4109 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [sys.catalog]: configured and running, proceeding with master startup.
I20250809 14:08:34.540426 4114 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdcde223499846178c470a88a31e0eba" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 46271 } }
I20250809 14:08:34.540879 4116 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdcde223499846178c470a88a31e0eba. Latest consensus state: current_term: 1 leader_uuid: "cdcde223499846178c470a88a31e0eba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdcde223499846178c470a88a31e0eba" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 46271 } } }
I20250809 14:08:34.540961 4116 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:34.540899 4115 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdcde223499846178c470a88a31e0eba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdcde223499846178c470a88a31e0eba" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 46271 } } }
I20250809 14:08:34.541015 4115 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:34.541162 4120 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250809 14:08:34.541683 4120 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250809 14:08:34.543767 4120 catalog_manager.cc:1349] Generated new cluster ID: 8329cdec63a84859a6c55a7ac0144fc9
I20250809 14:08:34.543843 4120 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250809 14:08:34.557965 4120 catalog_manager.cc:1372] Generated new certificate authority record
I20250809 14:08:34.558496 4120 catalog_manager.cc:1506] Loading token signing keys...
I20250809 14:08:34.565954 4120 catalog_manager.cc:5955] T 00000000000000000000000000000000 P cdcde223499846178c470a88a31e0eba: Generated new TSK 0
I20250809 14:08:34.566165 4120 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250809 14:08:34.574048 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.193:0
--local_ip_for_outbound_sockets=127.31.21.193
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:46271
--builtin_ntp_servers=127.31.21.212:37435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=30000 with env {}
W20250809 14:08:34.651732 4133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:34.651902 4133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:34.651932 4133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:34.653355 4133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:34.653412 4133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.193
I20250809 14:08:34.654846 4133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:37435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:46271
--never_fsync=true
--heap_profile_path=/tmp/kudu.4133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.193
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:34.655078 4133 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:34.655301 4133 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:34.657819 4141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:34.657819 4138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:34.657858 4139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:34.658046 4133 server_base.cc:1047] running on GCE node
I20250809 14:08:34.658349 4133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:34.658589 4133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:34.659761 4133 hybrid_clock.cc:648] HybridClock initialized: now 1754748514659742 us; error 32 us; skew 500 ppm
I20250809 14:08:34.661248 4133 webserver.cc:489] Webserver started at http://127.31.21.193:44137/ using document root <none> and password file <none>
I20250809 14:08:34.661518 4133 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:34.661581 4133 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:34.661705 4133 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:34.662851 4133 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "23d1d5bde8174117a9e6ca60a754d7af"
format_stamp: "Formatted at 2025-08-09 14:08:34 on dist-test-slave-mstq"
I20250809 14:08:34.663235 4133 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "23d1d5bde8174117a9e6ca60a754d7af"
format_stamp: "Formatted at 2025-08-09 14:08:34 on dist-test-slave-mstq"
I20250809 14:08:34.664723 4133 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20250809 14:08:34.665611 4147 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:34.665781 4133 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:34.665863 4133 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "23d1d5bde8174117a9e6ca60a754d7af"
format_stamp: "Formatted at 2025-08-09 14:08:34 on dist-test-slave-mstq"
I20250809 14:08:34.665927 4133 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:34.676059 4133 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:34.676297 4133 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:34.676429 4133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:34.676649 4133 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:34.677042 4133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:34.677088 4133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:34.677115 4133 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:34.677141 4133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:34.683650 4133 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.193:41007
I20250809 14:08:34.683722 4260 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.193:41007 every 8 connection(s)
I20250809 14:08:34.684015 4133 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20250809 14:08:34.688654 4261 heartbeater.cc:344] Connected to a master server at 127.31.21.254:46271
I20250809 14:08:34.688753 4261 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:34.688999 4261 heartbeater.cc:507] Master 127.31.21.254:46271 requested a full tablet report, sending...
I20250809 14:08:34.689087 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 4133
I20250809 14:08:34.689193 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.ListTableCliSimpleParamTest.TestListTables_2.1754748496992630-31831-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20250809 14:08:34.689435 4073 ts_manager.cc:194] Registered new tserver with Master: 23d1d5bde8174117a9e6ca60a754d7af (127.31.21.193:41007)
I20250809 14:08:34.690294 31831 external_mini_cluster.cc:949] 1 TS(s) registered with all masters
I20250809 14:08:34.690322 4073 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.193:38901
I20250809 14:08:34.696494 4073 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51920:
name: "TestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
owner: "alice"
I20250809 14:08:34.703015 4195 tablet_service.cc:1468] Processing CreateTablet for tablet 06ba626c20754b6fa0e62a969f3c4487 (DEFAULT_TABLE table=TestTable [id=4d26d7acec0b4b288873e4448c9917a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:34.703271 4195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06ba626c20754b6fa0e62a969f3c4487. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:34.705328 4271 tablet_bootstrap.cc:492] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af: Bootstrap starting.
I20250809 14:08:34.705900 4271 tablet_bootstrap.cc:654] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:34.706161 4271 log.cc:826] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:34.706746 4271 tablet_bootstrap.cc:492] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af: No bootstrap required, opened a new log
I20250809 14:08:34.706805 4271 ts_tablet_manager.cc:1397] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:34.707799 4271 raft_consensus.cc:357] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d1d5bde8174117a9e6ca60a754d7af" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 41007 } }
I20250809 14:08:34.707929 4271 raft_consensus.cc:383] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:34.707969 4271 raft_consensus.cc:738] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d1d5bde8174117a9e6ca60a754d7af, State: Initialized, Role: FOLLOWER
I20250809 14:08:34.708065 4271 consensus_queue.cc:260] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d1d5bde8174117a9e6ca60a754d7af" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 41007 } }
I20250809 14:08:34.708132 4271 raft_consensus.cc:397] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:34.708163 4271 raft_consensus.cc:491] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:34.708235 4271 raft_consensus.cc:3058] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:34.708724 4271 raft_consensus.cc:513] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d1d5bde8174117a9e6ca60a754d7af" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 41007 } }
I20250809 14:08:34.708817 4271 leader_election.cc:304] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23d1d5bde8174117a9e6ca60a754d7af; no voters:
I20250809 14:08:34.708976 4271 leader_election.cc:290] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:34.709045 4273 raft_consensus.cc:2802] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:34.709139 4271 ts_tablet_manager.cc:1428] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250809 14:08:34.709198 4273 raft_consensus.cc:695] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [term 1 LEADER]: Becoming Leader. State: Replica: 23d1d5bde8174117a9e6ca60a754d7af, State: Running, Role: LEADER
I20250809 14:08:34.709188 4261 heartbeater.cc:499] Master 127.31.21.254:46271 was elected leader, sending a full tablet report...
I20250809 14:08:34.709317 4273 consensus_queue.cc:237] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d1d5bde8174117a9e6ca60a754d7af" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 41007 } }
I20250809 14:08:34.710072 4073 catalog_manager.cc:5582] T 06ba626c20754b6fa0e62a969f3c4487 P 23d1d5bde8174117a9e6ca60a754d7af reported cstate change: term changed from 0 to 1, leader changed from <none> to 23d1d5bde8174117a9e6ca60a754d7af (127.31.21.193). New cstate: current_term: 1 leader_uuid: "23d1d5bde8174117a9e6ca60a754d7af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23d1d5bde8174117a9e6ca60a754d7af" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 41007 } health_report { overall_health: HEALTHY } } }
I20250809 14:08:34.712822 31831 external_mini_cluster.cc:949] 1 TS(s) registered with all masters
I20250809 14:08:34.713263 31831 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 23d1d5bde8174117a9e6ca60a754d7af to finish bootstrapping
I20250809 14:08:34.860013 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 4133
I20250809 14:08:34.864780 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 4042
2025-08-09T14:08:34Z chronyd exiting
[ OK ] ListTableCliSimpleParamTest.TestListTables/2 (465 ms)
[----------] 1 test from ListTableCliSimpleParamTest (465 ms total)
[----------] 1 test from ListTableCliParamTest
[ RUN ] ListTableCliParamTest.ListTabletWithPartitionInfo/4
I20250809 14:08:34.883040 31831 test_util.cc:276] Using random seed: 971396572
[ OK ] ListTableCliParamTest.ListTabletWithPartitionInfo/4 (1 ms)
[----------] 1 test from ListTableCliParamTest (1 ms total)
[----------] 1 test from IsSecure/SecureClusterAdminCliParamTest
[ RUN ] IsSecure/SecureClusterAdminCliParamTest.TestRebuildMaster/0
2025-08-09T14:08:34Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-08-09T14:08:34Z Disabled control of system clock
I20250809 14:08:34.892437 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:42875
--webserver_interface=127.31.21.254
--webserver_port=0
--builtin_ntp_servers=127.31.21.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.21.254:42875 with env {}
W20250809 14:08:34.971800 4295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:34.971979 4295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:34.972007 4295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:34.973373 4295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250809 14:08:34.973412 4295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:34.973426 4295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250809 14:08:34.973438 4295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250809 14:08:34.974869 4295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.21.254:42875
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:42875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.21.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.4295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:34.975061 4295 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:34.975251 4295 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:34.977777 4303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:34.977783 4300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:34.977949 4295 server_base.cc:1047] running on GCE node
W20250809 14:08:34.977782 4301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:34.978215 4295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:34.978447 4295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:34.979578 4295 hybrid_clock.cc:648] HybridClock initialized: now 1754748514979560 us; error 35 us; skew 500 ppm
I20250809 14:08:34.980683 4295 webserver.cc:489] Webserver started at http://127.31.21.254:41183/ using document root <none> and password file <none>
I20250809 14:08:34.980906 4295 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:34.980947 4295 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:34.981055 4295 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:34.981885 4295 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/instance:
uuid: "9b276abe7dfb4ecaae4dd07834d281c9"
format_stamp: "Formatted at 2025-08-09 14:08:34 on dist-test-slave-mstq"
I20250809 14:08:34.982182 4295 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal/instance:
uuid: "9b276abe7dfb4ecaae4dd07834d281c9"
format_stamp: "Formatted at 2025-08-09 14:08:34 on dist-test-slave-mstq"
I20250809 14:08:34.983283 4295 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250809 14:08:34.983956 4309 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:34.984127 4295 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250809 14:08:34.984200 4295 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
uuid: "9b276abe7dfb4ecaae4dd07834d281c9"
format_stamp: "Formatted at 2025-08-09 14:08:34 on dist-test-slave-mstq"
I20250809 14:08:34.984257 4295 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:35.017196 4295 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:35.017477 4295 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:35.017596 4295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:35.021191 4295 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.254:42875
I20250809 14:08:35.021239 4361 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.254:42875 every 8 connection(s)
I20250809 14:08:35.021550 4295 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/info.pb
I20250809 14:08:35.022143 4362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:35.024430 4362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9: Bootstrap starting.
I20250809 14:08:35.025048 4362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:35.025302 4362 log.cc:826] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:35.025898 4362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9: No bootstrap required, opened a new log
I20250809 14:08:35.026540 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 4295
I20250809 14:08:35.026628 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal/instance
I20250809 14:08:35.027160 4362 raft_consensus.cc:357] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b276abe7dfb4ecaae4dd07834d281c9" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 42875 } }
I20250809 14:08:35.027395 4362 raft_consensus.cc:383] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:35.027447 4362 raft_consensus.cc:738] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b276abe7dfb4ecaae4dd07834d281c9, State: Initialized, Role: FOLLOWER
I20250809 14:08:35.027542 4362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b276abe7dfb4ecaae4dd07834d281c9" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 42875 } }
I20250809 14:08:35.027612 4362 raft_consensus.cc:397] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:35.027647 4362 raft_consensus.cc:491] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:35.027688 4362 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:35.028282 4362 raft_consensus.cc:513] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b276abe7dfb4ecaae4dd07834d281c9" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 42875 } }
I20250809 14:08:35.028395 4362 leader_election.cc:304] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b276abe7dfb4ecaae4dd07834d281c9; no voters:
I20250809 14:08:35.028537 4362 leader_election.cc:290] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:35.028589 4367 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:35.028746 4362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [sys.catalog]: configured and running, proceeding with master startup.
I20250809 14:08:35.028745 4367 raft_consensus.cc:695] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [term 1 LEADER]: Becoming Leader. State: Replica: 9b276abe7dfb4ecaae4dd07834d281c9, State: Running, Role: LEADER
I20250809 14:08:35.028893 4367 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b276abe7dfb4ecaae4dd07834d281c9" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 42875 } }
I20250809 14:08:35.029232 4367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b276abe7dfb4ecaae4dd07834d281c9. Latest consensus state: current_term: 1 leader_uuid: "9b276abe7dfb4ecaae4dd07834d281c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b276abe7dfb4ecaae4dd07834d281c9" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 42875 } } }
I20250809 14:08:35.029242 4368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b276abe7dfb4ecaae4dd07834d281c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b276abe7dfb4ecaae4dd07834d281c9" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 42875 } } }
I20250809 14:08:35.030225 4367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:35.030239 4368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:35.030437 4383 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250809 14:08:35.031178 4383 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250809 14:08:35.032400 4383 catalog_manager.cc:1349] Generated new cluster ID: 0e171a212392492fa9ffef8956bed532
I20250809 14:08:35.032450 4383 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250809 14:08:35.040491 4383 catalog_manager.cc:1372] Generated new certificate authority record
I20250809 14:08:35.041163 4383 catalog_manager.cc:1506] Loading token signing keys...
I20250809 14:08:35.043720 4383 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 9b276abe7dfb4ecaae4dd07834d281c9: Generated new TSK 0
I20250809 14:08:35.043876 4383 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250809 14:08:35.047070 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.193:0
--local_ip_for_outbound_sockets=127.31.21.193
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:42875
--builtin_ntp_servers=127.31.21.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20250809 14:08:35.123207 4386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:35.123379 4386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:35.123400 4386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:35.124789 4386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:35.124877 4386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.193
I20250809 14:08:35.126363 4386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.21.193
--webserver_port=0
--tserver_master_addrs=127.31.21.254:42875
--never_fsync=true
--heap_profile_path=/tmp/kudu.4386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.193
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:35.126564 4386 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:35.126788 4386 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:35.129362 4392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:35.129436 4391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:35.129525 4386 server_base.cc:1047] running on GCE node
W20250809 14:08:35.129642 4394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:35.129806 4386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:35.130014 4386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:35.131143 4386 hybrid_clock.cc:648] HybridClock initialized: now 1754748515131121 us; error 44 us; skew 500 ppm
I20250809 14:08:35.132257 4386 webserver.cc:489] Webserver started at http://127.31.21.193:37437/ using document root <none> and password file <none>
I20250809 14:08:35.132448 4386 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:35.132493 4386 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:35.132617 4386 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:35.133543 4386 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data/instance:
uuid: "8a75602fc7db401e8fe48c4e4843e6f5"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:35.133853 4386 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal/instance:
uuid: "8a75602fc7db401e8fe48c4e4843e6f5"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:35.135018 4386 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20250809 14:08:35.135733 4400 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:35.135905 4386 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20250809 14:08:35.135977 4386 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal
uuid: "8a75602fc7db401e8fe48c4e4843e6f5"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:35.136039 4386 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:35.168962 4386 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:35.169219 4386 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:35.169337 4386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:35.169554 4386 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:35.169868 4386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:35.169904 4386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:35.169940 4386 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:35.169971 4386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:35.175343 4386 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.193:39899
I20250809 14:08:35.175415 4513 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.193:39899 every 8 connection(s)
I20250809 14:08:35.175693 4386 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data/info.pb
I20250809 14:08:35.180030 4514 heartbeater.cc:344] Connected to a master server at 127.31.21.254:42875
I20250809 14:08:35.180126 4514 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:35.180308 4514 heartbeater.cc:507] Master 127.31.21.254:42875 requested a full tablet report, sending...
I20250809 14:08:35.180732 4326 ts_manager.cc:194] Registered new tserver with Master: 8a75602fc7db401e8fe48c4e4843e6f5 (127.31.21.193:39899)
I20250809 14:08:35.181463 4326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.193:42249
I20250809 14:08:35.181687 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 4386
I20250809 14:08:35.181802 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal/instance
I20250809 14:08:35.182803 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.194:0
--local_ip_for_outbound_sockets=127.31.21.194
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:42875
--builtin_ntp_servers=127.31.21.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20250809 14:08:35.259382 4517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:35.259536 4517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:35.259565 4517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:35.260893 4517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:35.260950 4517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.194
I20250809 14:08:35.262369 4517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.21.194
--webserver_port=0
--tserver_master_addrs=127.31.21.254:42875
--never_fsync=true
--heap_profile_path=/tmp/kudu.4517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.194
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:35.262603 4517 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:35.262851 4517 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:35.265206 4523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:35.265256 4525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:35.265202 4522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:35.265306 4517 server_base.cc:1047] running on GCE node
I20250809 14:08:35.265617 4517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:35.265805 4517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:35.266938 4517 hybrid_clock.cc:648] HybridClock initialized: now 1754748515266919 us; error 35 us; skew 500 ppm
I20250809 14:08:35.267980 4517 webserver.cc:489] Webserver started at http://127.31.21.194:41901/ using document root <none> and password file <none>
I20250809 14:08:35.268173 4517 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:35.268213 4517 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:35.268309 4517 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:35.269155 4517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data/instance:
uuid: "3cdc59582437453090162beef9e1e4f3"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:35.269428 4517 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal/instance:
uuid: "3cdc59582437453090162beef9e1e4f3"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:35.270504 4517 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:35.271205 4531 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:35.271368 4517 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:35.271441 4517 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal
uuid: "3cdc59582437453090162beef9e1e4f3"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:35.271495 4517 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:35.284744 4517 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:35.285003 4517 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:35.285105 4517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:35.285297 4517 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:35.285559 4517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:35.285589 4517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:35.285611 4517 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:35.285638 4517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:35.290696 4517 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.194:43619
I20250809 14:08:35.290773 4644 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.194:43619 every 8 connection(s)
I20250809 14:08:35.291002 4517 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data/info.pb
I20250809 14:08:35.295174 4645 heartbeater.cc:344] Connected to a master server at 127.31.21.254:42875
I20250809 14:08:35.295260 4645 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:35.295434 4645 heartbeater.cc:507] Master 127.31.21.254:42875 requested a full tablet report, sending...
I20250809 14:08:35.295763 4326 ts_manager.cc:194] Registered new tserver with Master: 3cdc59582437453090162beef9e1e4f3 (127.31.21.194:43619)
I20250809 14:08:35.296253 4326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.194:33913
I20250809 14:08:35.297246 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 4517
I20250809 14:08:35.297329 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal/instance
I20250809 14:08:35.298228 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.195:0
--local_ip_for_outbound_sockets=127.31.21.195
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:42875
--builtin_ntp_servers=127.31.21.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20250809 14:08:35.376398 4648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:35.376574 4648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:35.376595 4648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:35.378137 4648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:35.378213 4648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.195
I20250809 14:08:35.379969 4648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.21.195
--webserver_port=0
--tserver_master_addrs=127.31.21.254:42875
--never_fsync=true
--heap_profile_path=/tmp/kudu.4648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.195
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:35.380223 4648 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:35.380460 4648 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:35.383157 4656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:35.383158 4654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:35.383221 4648 server_base.cc:1047] running on GCE node
W20250809 14:08:35.383158 4653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:35.383549 4648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:35.383737 4648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:35.384886 4648 hybrid_clock.cc:648] HybridClock initialized: now 1754748515384871 us; error 31 us; skew 500 ppm
I20250809 14:08:35.386026 4648 webserver.cc:489] Webserver started at http://127.31.21.195:42359/ using document root <none> and password file <none>
I20250809 14:08:35.386214 4648 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:35.386250 4648 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:35.386343 4648 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:35.387192 4648 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data/instance:
uuid: "182e7dfef1ff4a2eb4e8619be80c4e65"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:35.387465 4648 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal/instance:
uuid: "182e7dfef1ff4a2eb4e8619be80c4e65"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:35.388701 4648 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250809 14:08:35.389482 4662 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:35.389648 4648 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:35.389717 4648 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal
uuid: "182e7dfef1ff4a2eb4e8619be80c4e65"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:35.389782 4648 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:35.398411 4648 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:35.398661 4648 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:35.398772 4648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:35.398999 4648 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:35.399312 4648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250809 14:08:35.399344 4648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:35.399367 4648 ts_tablet_manager.cc:610] Registered 0 tablets
I20250809 14:08:35.399395 4648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:35.404716 4648 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.195:38585
I20250809 14:08:35.404783 4775 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.195:38585 every 8 connection(s)
I20250809 14:08:35.405119 4648 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data/info.pb
I20250809 14:08:35.409754 4776 heartbeater.cc:344] Connected to a master server at 127.31.21.254:42875
I20250809 14:08:35.409860 4776 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:35.410039 4776 heartbeater.cc:507] Master 127.31.21.254:42875 requested a full tablet report, sending...
I20250809 14:08:35.410410 4326 ts_manager.cc:194] Registered new tserver with Master: 182e7dfef1ff4a2eb4e8619be80c4e65 (127.31.21.195:38585)
I20250809 14:08:35.410759 4326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.195:57093
I20250809 14:08:35.412927 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 4648
I20250809 14:08:35.413020 31831 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal/instance
I20250809 14:08:35.414100 31831 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250809 14:08:35.418905 31831 test_util.cc:276] Using random seed: 971932438
I20250809 14:08:35.424437 4326 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48768:
name: "pre_rebuild"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250809 14:08:35.424736 4326 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table pre_rebuild in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250809 14:08:35.431299 4579 tablet_service.cc:1468] Processing CreateTablet for tablet f09f57826d7f4bb2be4fd10ea94a2a38 (DEFAULT_TABLE table=pre_rebuild [id=1765db1663d3422c94410e8f4c75a527]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:35.431299 4448 tablet_service.cc:1468] Processing CreateTablet for tablet f09f57826d7f4bb2be4fd10ea94a2a38 (DEFAULT_TABLE table=pre_rebuild [id=1765db1663d3422c94410e8f4c75a527]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:35.431651 4579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f09f57826d7f4bb2be4fd10ea94a2a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:35.431651 4448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f09f57826d7f4bb2be4fd10ea94a2a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:35.434530 4795 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Bootstrap starting.
I20250809 14:08:35.434923 4710 tablet_service.cc:1468] Processing CreateTablet for tablet f09f57826d7f4bb2be4fd10ea94a2a38 (DEFAULT_TABLE table=pre_rebuild [id=1765db1663d3422c94410e8f4c75a527]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:35.435192 4710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f09f57826d7f4bb2be4fd10ea94a2a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:35.435238 4795 tablet_bootstrap.cc:654] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:35.435499 4795 log.cc:826] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:35.435809 4796 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Bootstrap starting.
I20250809 14:08:35.436223 4795 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: No bootstrap required, opened a new log
I20250809 14:08:35.436293 4795 ts_tablet_manager.cc:1397] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:35.436501 4796 tablet_bootstrap.cc:654] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:35.436764 4796 log.cc:826] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:35.437397 4796 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: No bootstrap required, opened a new log
I20250809 14:08:35.437460 4796 ts_tablet_manager.cc:1397] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250809 14:08:35.437755 4799 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Bootstrap starting.
I20250809 14:08:35.438015 4795 raft_consensus.cc:357] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:35.438148 4795 raft_consensus.cc:383] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:35.438179 4795 raft_consensus.cc:738] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cdc59582437453090162beef9e1e4f3, State: Initialized, Role: FOLLOWER
I20250809 14:08:35.438287 4795 consensus_queue.cc:260] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:35.438397 4799 tablet_bootstrap.cc:654] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:35.438539 4795 ts_tablet_manager.cc:1428] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:35.438663 4645 heartbeater.cc:499] Master 127.31.21.254:42875 was elected leader, sending a full tablet report...
I20250809 14:08:35.438736 4799 log.cc:826] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:35.439116 4796 raft_consensus.cc:357] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:35.439261 4796 raft_consensus.cc:383] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:35.439304 4796 raft_consensus.cc:738] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a75602fc7db401e8fe48c4e4843e6f5, State: Initialized, Role: FOLLOWER
I20250809 14:08:35.439395 4796 consensus_queue.cc:260] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:35.439631 4796 ts_tablet_manager.cc:1428] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:35.439771 4514 heartbeater.cc:499] Master 127.31.21.254:42875 was elected leader, sending a full tablet report...
I20250809 14:08:35.440235 4799 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: No bootstrap required, opened a new log
I20250809 14:08:35.440304 4799 ts_tablet_manager.cc:1397] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250809 14:08:35.441478 4799 raft_consensus.cc:357] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:35.441591 4799 raft_consensus.cc:383] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:35.441619 4799 raft_consensus.cc:738] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182e7dfef1ff4a2eb4e8619be80c4e65, State: Initialized, Role: FOLLOWER
I20250809 14:08:35.441713 4799 consensus_queue.cc:260] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:35.441967 4799 ts_tablet_manager.cc:1428] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250809 14:08:35.442018 4776 heartbeater.cc:499] Master 127.31.21.254:42875 was elected leader, sending a full tablet report...
I20250809 14:08:35.480263 4803 raft_consensus.cc:491] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250809 14:08:35.480393 4803 raft_consensus.cc:513] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:35.480734 4803 leader_election.cc:290] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a75602fc7db401e8fe48c4e4843e6f5 (127.31.21.193:39899), 3cdc59582437453090162beef9e1e4f3 (127.31.21.194:43619)
I20250809 14:08:35.483419 4599 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f09f57826d7f4bb2be4fd10ea94a2a38" candidate_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cdc59582437453090162beef9e1e4f3" is_pre_election: true
I20250809 14:08:35.483564 4599 raft_consensus.cc:2466] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182e7dfef1ff4a2eb4e8619be80c4e65 in term 0.
I20250809 14:08:35.483736 4468 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f09f57826d7f4bb2be4fd10ea94a2a38" candidate_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" is_pre_election: true
I20250809 14:08:35.483860 4468 raft_consensus.cc:2466] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182e7dfef1ff4a2eb4e8619be80c4e65 in term 0.
I20250809 14:08:35.484066 4663 leader_election.cc:304] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182e7dfef1ff4a2eb4e8619be80c4e65, 3cdc59582437453090162beef9e1e4f3; no voters:
I20250809 14:08:35.484180 4803 raft_consensus.cc:2802] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250809 14:08:35.484228 4803 raft_consensus.cc:491] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250809 14:08:35.484258 4803 raft_consensus.cc:3058] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:35.484838 4803 raft_consensus.cc:513] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:35.484987 4803 leader_election.cc:290] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [CANDIDATE]: Term 1 election: Requested vote from peers 8a75602fc7db401e8fe48c4e4843e6f5 (127.31.21.193:39899), 3cdc59582437453090162beef9e1e4f3 (127.31.21.194:43619)
I20250809 14:08:35.485167 4468 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f09f57826d7f4bb2be4fd10ea94a2a38" candidate_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a75602fc7db401e8fe48c4e4843e6f5"
I20250809 14:08:35.485157 4599 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f09f57826d7f4bb2be4fd10ea94a2a38" candidate_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cdc59582437453090162beef9e1e4f3"
I20250809 14:08:35.485235 4468 raft_consensus.cc:3058] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:35.485246 4599 raft_consensus.cc:3058] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:35.485870 4468 raft_consensus.cc:2466] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182e7dfef1ff4a2eb4e8619be80c4e65 in term 1.
I20250809 14:08:35.485927 4599 raft_consensus.cc:2466] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182e7dfef1ff4a2eb4e8619be80c4e65 in term 1.
I20250809 14:08:35.486093 4666 leader_election.cc:304] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182e7dfef1ff4a2eb4e8619be80c4e65, 8a75602fc7db401e8fe48c4e4843e6f5; no voters:
I20250809 14:08:35.486232 4803 raft_consensus.cc:2802] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:35.486374 4803 raft_consensus.cc:695] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 1 LEADER]: Becoming Leader. State: Replica: 182e7dfef1ff4a2eb4e8619be80c4e65, State: Running, Role: LEADER
I20250809 14:08:35.486490 4803 consensus_queue.cc:237] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:35.487246 4325 catalog_manager.cc:5582] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182e7dfef1ff4a2eb4e8619be80c4e65 (127.31.21.195). New cstate: current_term: 1 leader_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } health_report { overall_health: HEALTHY } } }
I20250809 14:08:35.513833 4468 raft_consensus.cc:1273] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 1 FOLLOWER]: Refusing update from remote peer 182e7dfef1ff4a2eb4e8619be80c4e65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250809 14:08:35.513839 4599 raft_consensus.cc:1273] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 1 FOLLOWER]: Refusing update from remote peer 182e7dfef1ff4a2eb4e8619be80c4e65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250809 14:08:35.514216 4805 consensus_queue.cc:1035] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250809 14:08:35.514333 4803 consensus_queue.cc:1035] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250809 14:08:35.519169 4814 mvcc.cc:204] Tried to move back new op lower bound from 7187449919544217600 to 7187449919433334784. Current Snapshot: MvccSnapshot[applied={T|T < 7187449919542353920}]
I20250809 14:08:35.682335 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 4295
W20250809 14:08:35.765934 4834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:35.766108 4834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:35.767482 4834 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
I20250809 14:08:35.785341 4834 minidump.cc:252] Setting minidump size limit to 20M
I20250809 14:08:35.785657 4834 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:35.785867 4834 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:35.788050 4855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:35.788147 4858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:35.788180 4856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:35.788183 4834 server_base.cc:1047] running on GCE node
I20250809 14:08:35.788446 4834 hybrid_clock.cc:584] initializing the hybrid clock with 'system' time source
I20250809 14:08:35.788504 4834 hybrid_clock.cc:648] HybridClock initialized: now 1754748515788503 us; error 106712 us; skew 500 ppm
I20250809 14:08:35.789290 4834 webserver.cc:489] Webserver started at http://0.0.0.0:39293/ using document root <none> and password file <none>
I20250809 14:08:35.789458 4834 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:35.789507 4834 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:35.789602 4834 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250809 14:08:35.790360 4834 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/instance:
uuid: "da6e2529de77434fbe5d1474ba8eb8c3"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:35.790629 4834 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal/instance:
uuid: "da6e2529de77434fbe5d1474ba8eb8c3"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:35.791702 4834 fs_manager.cc:696] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20250809 14:08:35.792337 4863 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:35.792467 4834 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:35.792537 4834 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
uuid: "da6e2529de77434fbe5d1474ba8eb8c3"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:35.792598 4834 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:35.911823 4834 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:35.912146 4834 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:35.912276 4834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:35.912777 4834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:35.914978 4834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3: Bootstrap starting.
I20250809 14:08:35.915508 4834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:35.915745 4834 log.cc:826] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:35.916319 4834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3: No bootstrap required, opened a new log
I20250809 14:08:35.917534 4834 raft_consensus.cc:357] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" member_type: VOTER }
I20250809 14:08:35.917637 4834 raft_consensus.cc:383] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:35.917656 4834 raft_consensus.cc:738] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da6e2529de77434fbe5d1474ba8eb8c3, State: Initialized, Role: FOLLOWER
I20250809 14:08:35.917733 4834 consensus_queue.cc:260] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" member_type: VOTER }
I20250809 14:08:35.917775 4834 raft_consensus.cc:397] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:35.917799 4834 raft_consensus.cc:491] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:35.917829 4834 raft_consensus.cc:3058] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:35.918262 4834 raft_consensus.cc:513] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" member_type: VOTER }
I20250809 14:08:35.918344 4834 leader_election.cc:304] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da6e2529de77434fbe5d1474ba8eb8c3; no voters:
I20250809 14:08:35.918459 4834 leader_election.cc:290] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250809 14:08:35.918531 4870 raft_consensus.cc:2802] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:35.918674 4870 raft_consensus.cc:695] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 1 LEADER]: Becoming Leader. State: Replica: da6e2529de77434fbe5d1474ba8eb8c3, State: Running, Role: LEADER
I20250809 14:08:35.918761 4870 consensus_queue.cc:237] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" member_type: VOTER }
I20250809 14:08:35.919134 4872 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da6e2529de77434fbe5d1474ba8eb8c3. Latest consensus state: current_term: 1 leader_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" member_type: VOTER } }
I20250809 14:08:35.919181 4871 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" member_type: VOTER } }
I20250809 14:08:35.919260 4872 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:35.919340 4871 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:35.920403 4834 tablet_replica.cc:331] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3: stopping tablet replica
I20250809 14:08:35.920470 4834 raft_consensus.cc:2241] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 1 LEADER]: Raft consensus shutting down.
I20250809 14:08:35.920522 4834 raft_consensus.cc:2270] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250809 14:08:35.920845 4834 master.cc:561] Master@0.0.0.0:7051 shutting down...
W20250809 14:08:35.920969 4834 acceptor_pool.cc:194] Could not shut down acceptor socket on 0.0.0.0:7051: Network error: shutdown error: Transport endpoint is not connected (error 107)
I20250809 14:08:35.931850 4834 master.cc:583] Master@0.0.0.0:7051 shutdown complete.
I20250809 14:08:35.938153 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 4386
I20250809 14:08:35.943045 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 4517
I20250809 14:08:35.947657 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 4648
I20250809 14:08:35.953307 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:42875
--webserver_interface=127.31.21.254
--webserver_port=41183
--builtin_ntp_servers=127.31.21.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.21.254:42875 with env {}
W20250809 14:08:36.029659 4876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:36.029829 4876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:36.029846 4876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:36.031217 4876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250809 14:08:36.031260 4876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:36.031273 4876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250809 14:08:36.031287 4876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250809 14:08:36.032754 4876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.21.254:42875
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.21.254:42875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.21.254
--webserver_port=41183
--never_fsync=true
--heap_profile_path=/tmp/kudu.4876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:36.033021 4876 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:36.033262 4876 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:36.035678 4881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:36.035660 4876 server_base.cc:1047] running on GCE node
W20250809 14:08:36.035725 4884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:36.035790 4882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:36.036123 4876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:36.036343 4876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:36.037468 4876 hybrid_clock.cc:648] HybridClock initialized: now 1754748516037447 us; error 38 us; skew 500 ppm
I20250809 14:08:36.038559 4876 webserver.cc:489] Webserver started at http://127.31.21.254:41183/ using document root <none> and password file <none>
I20250809 14:08:36.038762 4876 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:36.038821 4876 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:36.039975 4876 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:36.040556 4890 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:36.040741 4876 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:36.040812 4876 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
uuid: "da6e2529de77434fbe5d1474ba8eb8c3"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:36.041111 4876 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:36.049269 4876 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:36.049507 4876 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:36.049613 4876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:36.053231 4876 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.254:42875
I20250809 14:08:36.053269 4942 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.254:42875 every 8 connection(s)
I20250809 14:08:36.053573 4876 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/info.pb
I20250809 14:08:36.054550 4943 sys_catalog.cc:263] Verifying existing consensus state
I20250809 14:08:36.056440 4943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3: Bootstrap starting.
I20250809 14:08:36.057456 4943 log.cc:826] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:36.057751 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 4876
I20250809 14:08:36.058029 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.193:39899
--local_ip_for_outbound_sockets=127.31.21.193
--tserver_master_addrs=127.31.21.254:42875
--webserver_port=37437
--webserver_interface=127.31.21.193
--builtin_ntp_servers=127.31.21.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20250809 14:08:36.058815 4943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=2 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:36.059041 4943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3: Bootstrap complete.
I20250809 14:08:36.060210 4943 raft_consensus.cc:357] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 42875 } }
I20250809 14:08:36.060325 4943 raft_consensus.cc:738] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da6e2529de77434fbe5d1474ba8eb8c3, State: Initialized, Role: FOLLOWER
I20250809 14:08:36.060432 4943 consensus_queue.cc:260] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 42875 } }
I20250809 14:08:36.060488 4943 raft_consensus.cc:397] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250809 14:08:36.060525 4943 raft_consensus.cc:491] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250809 14:08:36.060571 4943 raft_consensus.cc:3058] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 1 FOLLOWER]: Advancing to term 2
I20250809 14:08:36.061128 4943 raft_consensus.cc:513] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 42875 } }
I20250809 14:08:36.061216 4943 leader_election.cc:304] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da6e2529de77434fbe5d1474ba8eb8c3; no voters:
I20250809 14:08:36.061350 4943 leader_election.cc:290] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20250809 14:08:36.061407 4947 raft_consensus.cc:2802] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 2 FOLLOWER]: Leader election won for term 2
I20250809 14:08:36.061522 4947 raft_consensus.cc:695] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [term 2 LEADER]: Becoming Leader. State: Replica: da6e2529de77434fbe5d1474ba8eb8c3, State: Running, Role: LEADER
I20250809 14:08:36.061578 4943 sys_catalog.cc:564] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [sys.catalog]: configured and running, proceeding with master startup.
I20250809 14:08:36.061650 4947 consensus_queue.cc:237] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 42875 } }
I20250809 14:08:36.061905 4947 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da6e2529de77434fbe5d1474ba8eb8c3. Latest consensus state: current_term: 2 leader_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 42875 } } }
I20250809 14:08:36.061973 4947 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:36.062366 4952 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250809 14:08:36.063031 4952 catalog_manager.cc:671] Loaded metadata for table pre_rebuild [id=c9f1c34f4ce247a6a8fa52596e6d514d]
I20250809 14:08:36.063319 4952 tablet_loader.cc:96] loaded metadata for tablet f09f57826d7f4bb2be4fd10ea94a2a38 (table pre_rebuild [id=c9f1c34f4ce247a6a8fa52596e6d514d])
I20250809 14:08:36.063382 4952 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250809 14:08:36.061944 4949 sys_catalog.cc:455] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da6e2529de77434fbe5d1474ba8eb8c3" member_type: VOTER last_known_addr { host: "127.31.21.254" port: 42875 } } }
I20250809 14:08:36.063966 4949 sys_catalog.cc:458] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3 [sys.catalog]: This master's current role is: LEADER
I20250809 14:08:36.064411 4952 catalog_manager.cc:1349] Generated new cluster ID: ddd9e5beb38944e9acdb36011e5d23dd
I20250809 14:08:36.064455 4952 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250809 14:08:36.068678 4952 catalog_manager.cc:1372] Generated new certificate authority record
I20250809 14:08:36.069237 4952 catalog_manager.cc:1506] Loading token signing keys...
I20250809 14:08:36.074731 4952 catalog_manager.cc:5955] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3: Generated new TSK 0
I20250809 14:08:36.074894 4952 catalog_manager.cc:1516] Initializing in-progress tserver states...
W20250809 14:08:36.141716 4946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:36.141883 4946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:36.141902 4946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:36.143357 4946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:36.143409 4946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.193
I20250809 14:08:36.144956 4946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.193:39899
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.21.193
--webserver_port=37437
--tserver_master_addrs=127.31.21.254:42875
--never_fsync=true
--heap_profile_path=/tmp/kudu.4946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.193
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:36.145149 4946 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:36.145360 4946 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:36.147630 4973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:36.147634 4970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:36.147790 4946 server_base.cc:1047] running on GCE node
W20250809 14:08:36.147641 4971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:36.148129 4946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:36.148331 4946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:36.149467 4946 hybrid_clock.cc:648] HybridClock initialized: now 1754748516149446 us; error 38 us; skew 500 ppm
I20250809 14:08:36.150570 4946 webserver.cc:489] Webserver started at http://127.31.21.193:37437/ using document root <none> and password file <none>
I20250809 14:08:36.150759 4946 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:36.150820 4946 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:36.152034 4946 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:36.152729 4979 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:36.152922 4946 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:36.152997 4946 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal
uuid: "8a75602fc7db401e8fe48c4e4843e6f5"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:36.153254 4946 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:36.163599 4946 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:36.163847 4946 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:36.163968 4946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:36.164170 4946 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:36.164594 4986 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250809 14:08:36.165746 4946 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250809 14:08:36.165792 4946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:36.165836 4946 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250809 14:08:36.166379 4946 ts_tablet_manager.cc:610] Registered 1 tablets
I20250809 14:08:36.166411 4946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:36.166500 4986 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Bootstrap starting.
I20250809 14:08:36.173384 4946 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.193:39899
I20250809 14:08:36.173714 4946 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data/info.pb
I20250809 14:08:36.177064 5093 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.193:39899 every 8 connection(s)
I20250809 14:08:36.179414 5094 heartbeater.cc:344] Connected to a master server at 127.31.21.254:42875
I20250809 14:08:36.179512 5094 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:36.179700 5094 heartbeater.cc:507] Master 127.31.21.254:42875 requested a full tablet report, sending...
I20250809 14:08:36.180313 4907 ts_manager.cc:194] Registered new tserver with Master: 8a75602fc7db401e8fe48c4e4843e6f5 (127.31.21.193:39899)
I20250809 14:08:36.180819 4986 log.cc:826] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:36.181156 4907 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.193:56047
I20250809 14:08:36.182672 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 4946
I20250809 14:08:36.182931 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.194:43619
--local_ip_for_outbound_sockets=127.31.21.194
--tserver_master_addrs=127.31.21.254:42875
--webserver_port=41901
--webserver_interface=127.31.21.194
--builtin_ntp_servers=127.31.21.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20250809 14:08:36.229844 4986 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=214 overwritten=0 applied=214 ignored=0} inserts{seen=10650 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:36.230250 4986 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Bootstrap complete.
I20250809 14:08:36.231454 4986 ts_tablet_manager.cc:1397] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Time spent bootstrapping tablet: real 0.065s user 0.058s sys 0.004s
I20250809 14:08:36.232643 4986 raft_consensus.cc:357] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:36.232988 4986 raft_consensus.cc:738] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a75602fc7db401e8fe48c4e4843e6f5, State: Initialized, Role: FOLLOWER
I20250809 14:08:36.233163 4986 consensus_queue.cc:260] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 214, Last appended: 1.214, Last appended by leader: 214, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:36.233479 4986 ts_tablet_manager.cc:1428] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20250809 14:08:36.233522 5094 heartbeater.cc:499] Master 127.31.21.254:42875 was elected leader, sending a full tablet report...
W20250809 14:08:36.260308 5099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:36.260476 5099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:36.260507 5099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:36.261914 5099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:36.261976 5099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.194
I20250809 14:08:36.263640 5099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.194:43619
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.21.194
--webserver_port=41901
--tserver_master_addrs=127.31.21.254:42875
--never_fsync=true
--heap_profile_path=/tmp/kudu.5099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.194
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:36.263873 5099 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:36.264108 5099 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:36.266693 5106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:36.266732 5108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:36.266693 5105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:36.266903 5099 server_base.cc:1047] running on GCE node
I20250809 14:08:36.267117 5099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:36.267295 5099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:36.268419 5099 hybrid_clock.cc:648] HybridClock initialized: now 1754748516268410 us; error 28 us; skew 500 ppm
I20250809 14:08:36.269532 5099 webserver.cc:489] Webserver started at http://127.31.21.194:41901/ using document root <none> and password file <none>
I20250809 14:08:36.269726 5099 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:36.269779 5099 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:36.270915 5099 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:36.271484 5114 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:36.271636 5099 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20250809 14:08:36.271705 5099 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal
uuid: "3cdc59582437453090162beef9e1e4f3"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:36.271953 5099 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:36.291791 5099 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:36.292063 5099 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:36.292215 5099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:36.292439 5099 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:36.292922 5121 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250809 14:08:36.293747 5099 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250809 14:08:36.293797 5099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:36.293853 5099 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250809 14:08:36.294413 5099 ts_tablet_manager.cc:610] Registered 1 tablets
I20250809 14:08:36.294445 5099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:36.294504 5121 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Bootstrap starting.
I20250809 14:08:36.301421 5099 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.194:43619
I20250809 14:08:36.301501 5228 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.194:43619 every 8 connection(s)
I20250809 14:08:36.301784 5099 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data/info.pb
I20250809 14:08:36.305526 5121 log.cc:826] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:36.306303 5229 heartbeater.cc:344] Connected to a master server at 127.31.21.254:42875
I20250809 14:08:36.306421 5229 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:36.306604 5229 heartbeater.cc:507] Master 127.31.21.254:42875 requested a full tablet report, sending...
I20250809 14:08:36.307107 4907 ts_manager.cc:194] Registered new tserver with Master: 3cdc59582437453090162beef9e1e4f3 (127.31.21.194:43619)
I20250809 14:08:36.307502 4907 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.194:52715
I20250809 14:08:36.307500 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 5099
I20250809 14:08:36.307781 31831 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskwLxUSP/build/release/bin/kudu
/tmp/dist-test-taskwLxUSP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.21.195:38585
--local_ip_for_outbound_sockets=127.31.21.195
--tserver_master_addrs=127.31.21.254:42875
--webserver_port=42359
--webserver_interface=127.31.21.195
--builtin_ntp_servers=127.31.21.212:35329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20250809 14:08:36.335778 5121 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=214 overwritten=0 applied=214 ignored=0} inserts{seen=10650 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:36.336127 5121 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Bootstrap complete.
I20250809 14:08:36.337052 5121 ts_tablet_manager.cc:1397] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Time spent bootstrapping tablet: real 0.043s user 0.031s sys 0.011s
I20250809 14:08:36.337867 5121 raft_consensus.cc:357] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:36.338056 5121 raft_consensus.cc:738] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cdc59582437453090162beef9e1e4f3, State: Initialized, Role: FOLLOWER
I20250809 14:08:36.338145 5121 consensus_queue.cc:260] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 214, Last appended: 1.214, Last appended by leader: 214, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:36.338394 5121 ts_tablet_manager.cc:1428] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:36.338479 5229 heartbeater.cc:499] Master 127.31.21.254:42875 was elected leader, sending a full tablet report...
W20250809 14:08:36.385160 5233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250809 14:08:36.385332 5233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250809 14:08:36.385363 5233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250809 14:08:36.386723 5233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250809 14:08:36.386781 5233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.21.195
I20250809 14:08:36.388255 5233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.21.212:35329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.21.195:38585
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.21.195
--webserver_port=42359
--tserver_master_addrs=127.31.21.254:42875
--never_fsync=true
--heap_profile_path=/tmp/kudu.5233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.21.195
--log_dir=/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision b92f16d1c86a753c597b46c7575bfa6a1479726a
build type RELEASE
built by None at 09 Aug 2025 13:43:12 UTC on 5fd53c4cbb9d
build id 7483
I20250809 14:08:36.388492 5233 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250809 14:08:36.388749 5233 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250809 14:08:36.391363 5240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:36.391373 5242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250809 14:08:36.391439 5239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250809 14:08:36.391455 5233 server_base.cc:1047] running on GCE node
I20250809 14:08:36.391688 5233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250809 14:08:36.391883 5233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250809 14:08:36.393010 5233 hybrid_clock.cc:648] HybridClock initialized: now 1754748516392991 us; error 40 us; skew 500 ppm
I20250809 14:08:36.394047 5233 webserver.cc:489] Webserver started at http://127.31.21.195:42359/ using document root <none> and password file <none>
I20250809 14:08:36.394237 5233 fs_manager.cc:362] Metadata directory not provided
I20250809 14:08:36.394304 5233 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250809 14:08:36.395450 5233 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:36.396011 5248 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:36.396166 5233 fs_manager.cc:730] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:36.396234 5233 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal
uuid: "182e7dfef1ff4a2eb4e8619be80c4e65"
format_stamp: "Formatted at 2025-08-09 14:08:35 on dist-test-slave-mstq"
I20250809 14:08:36.396481 5233 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250809 14:08:36.423765 5233 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250809 14:08:36.424034 5233 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250809 14:08:36.424154 5233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250809 14:08:36.424378 5233 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250809 14:08:36.424793 5255 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20250809 14:08:36.425663 5233 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20250809 14:08:36.425707 5233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:36.425745 5233 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20250809 14:08:36.426313 5233 ts_tablet_manager.cc:610] Registered 1 tablets
I20250809 14:08:36.426343 5233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20250809 14:08:36.426403 5255 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Bootstrap starting.
I20250809 14:08:36.432435 5233 rpc_server.cc:307] RPC server started. Bound to: 127.31.21.195:38585
I20250809 14:08:36.432487 5362 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.31.21.195:38585 every 8 connection(s)
I20250809 14:08:36.432771 5233 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data/info.pb
I20250809 14:08:36.438166 5363 heartbeater.cc:344] Connected to a master server at 127.31.21.254:42875
I20250809 14:08:36.438290 5363 heartbeater.cc:461] Registering TS with master...
I20250809 14:08:36.438548 5363 heartbeater.cc:507] Master 127.31.21.254:42875 requested a full tablet report, sending...
I20250809 14:08:36.439152 4907 ts_manager.cc:194] Registered new tserver with Master: 182e7dfef1ff4a2eb4e8619be80c4e65 (127.31.21.195:38585)
I20250809 14:08:36.439591 5255 log.cc:826] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Log is configured to *not* fsync() on all Append() calls
I20250809 14:08:36.439674 4907 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.21.195:37935
I20250809 14:08:36.442826 31831 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskwLxUSP/build/release/bin/kudu as pid 5233
I20250809 14:08:36.447059 31831 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20250809 14:08:36.472679 5255 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Bootstrap replayed 1/1 log segments. Stats: ops{read=214 overwritten=0 applied=214 ignored=0} inserts{seen=10650 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250809 14:08:36.473047 5255 tablet_bootstrap.cc:492] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Bootstrap complete.
I20250809 14:08:36.473888 5255 ts_tablet_manager.cc:1397] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Time spent bootstrapping tablet: real 0.048s user 0.035s sys 0.011s
I20250809 14:08:36.474715 5255 raft_consensus.cc:357] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:36.474911 5255 raft_consensus.cc:738] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182e7dfef1ff4a2eb4e8619be80c4e65, State: Initialized, Role: FOLLOWER
I20250809 14:08:36.475005 5255 consensus_queue.cc:260] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 214, Last appended: 1.214, Last appended by leader: 214, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:36.475273 5255 ts_tablet_manager.cc:1428] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Time spent starting tablet: real 0.001s user 0.002s sys 0.001s
I20250809 14:08:36.475344 5363 heartbeater.cc:499] Master 127.31.21.254:42875 was elected leader, sending a full tablet report...
I20250809 14:08:37.618971 5374 raft_consensus.cc:491] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250809 14:08:37.619114 5374 raft_consensus.cc:513] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:37.619416 5374 leader_election.cc:290] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3cdc59582437453090162beef9e1e4f3 (127.31.21.194:43619), 182e7dfef1ff4a2eb4e8619be80c4e65 (127.31.21.195:38585)
I20250809 14:08:37.622848 5183 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f09f57826d7f4bb2be4fd10ea94a2a38" candidate_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 214 } } ignore_live_leader: false dest_uuid: "3cdc59582437453090162beef9e1e4f3" is_pre_election: true
I20250809 14:08:37.622841 5317 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f09f57826d7f4bb2be4fd10ea94a2a38" candidate_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 214 } } ignore_live_leader: false dest_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" is_pre_election: true
I20250809 14:08:37.622973 5183 raft_consensus.cc:2466] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a75602fc7db401e8fe48c4e4843e6f5 in term 1.
I20250809 14:08:37.622977 5317 raft_consensus.cc:2466] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a75602fc7db401e8fe48c4e4843e6f5 in term 1.
I20250809 14:08:37.623229 4981 leader_election.cc:304] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182e7dfef1ff4a2eb4e8619be80c4e65, 8a75602fc7db401e8fe48c4e4843e6f5; no voters:
I20250809 14:08:37.623369 5374 raft_consensus.cc:2802] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250809 14:08:37.623445 5374 raft_consensus.cc:491] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250809 14:08:37.623464 5374 raft_consensus.cc:3058] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 1 FOLLOWER]: Advancing to term 2
I20250809 14:08:37.624423 5374 raft_consensus.cc:513] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:37.624539 5374 leader_election.cc:290] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [CANDIDATE]: Term 2 election: Requested vote from peers 3cdc59582437453090162beef9e1e4f3 (127.31.21.194:43619), 182e7dfef1ff4a2eb4e8619be80c4e65 (127.31.21.195:38585)
I20250809 14:08:37.624697 5317 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f09f57826d7f4bb2be4fd10ea94a2a38" candidate_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 214 } } ignore_live_leader: false dest_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65"
I20250809 14:08:37.624699 5183 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f09f57826d7f4bb2be4fd10ea94a2a38" candidate_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 214 } } ignore_live_leader: false dest_uuid: "3cdc59582437453090162beef9e1e4f3"
I20250809 14:08:37.624773 5183 raft_consensus.cc:3058] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 1 FOLLOWER]: Advancing to term 2
I20250809 14:08:37.624773 5317 raft_consensus.cc:3058] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 1 FOLLOWER]: Advancing to term 2
I20250809 14:08:37.625994 5317 raft_consensus.cc:2466] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a75602fc7db401e8fe48c4e4843e6f5 in term 2.
I20250809 14:08:37.625994 5183 raft_consensus.cc:2466] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a75602fc7db401e8fe48c4e4843e6f5 in term 2.
I20250809 14:08:37.626185 4981 leader_election.cc:304] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182e7dfef1ff4a2eb4e8619be80c4e65, 8a75602fc7db401e8fe48c4e4843e6f5; no voters:
I20250809 14:08:37.626273 5374 raft_consensus.cc:2802] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 2 FOLLOWER]: Leader election won for term 2
I20250809 14:08:37.626380 5374 raft_consensus.cc:695] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 2 LEADER]: Becoming Leader. State: Replica: 8a75602fc7db401e8fe48c4e4843e6f5, State: Running, Role: LEADER
I20250809 14:08:37.626459 5374 consensus_queue.cc:237] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 214, Committed index: 214, Last appended: 1.214, Last appended by leader: 214, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:37.627149 4907 catalog_manager.cc:5582] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 reported cstate change: term changed from 0 to 2, leader changed from <none> to 8a75602fc7db401e8fe48c4e4843e6f5 (127.31.21.193), VOTER 182e7dfef1ff4a2eb4e8619be80c4e65 (127.31.21.195) added, VOTER 3cdc59582437453090162beef9e1e4f3 (127.31.21.194) added, VOTER 8a75602fc7db401e8fe48c4e4843e6f5 (127.31.21.193) added. New cstate: current_term: 2 leader_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } health_report { overall_health: UNKNOWN } } }
W20250809 14:08:37.967665 31831 scanner-internal.cc:458] Time spent opening tablet: real 1.515s user 0.001s sys 0.001s
I20250809 14:08:38.010644 5297 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:38.011421 5028 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250809 14:08:38.015185 5163 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
da6e2529de77434fbe5d1474ba8eb8c3 | 127.31.21.254:42875 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.31.21.212:35329 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
182e7dfef1ff4a2eb4e8619be80c4e65 | 127.31.21.195:38585 | HEALTHY | <none> | 0 | 0
3cdc59582437453090162beef9e1e4f3 | 127.31.21.194:43619 | HEALTHY | <none> | 0 | 0
8a75602fc7db401e8fe48c4e4843e6f5 | 127.31.21.193:39899 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.31.21.193 | experimental | 127.31.21.193:39899
local_ip_for_outbound_sockets | 127.31.21.194 | experimental | 127.31.21.194:43619
local_ip_for_outbound_sockets | 127.31.21.195 | experimental | 127.31.21.195:38585
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data/info.pb | hidden | 127.31.21.193:39899
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data/info.pb | hidden | 127.31.21.194:43619
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data/info.pb | hidden | 127.31.21.195:38585
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.31.21.212:35329 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet f09f57826d7f4bb2be4fd10ea94a2a38 of table 'pre_rebuild' is conflicted: 2 replicas' active configs disagree with the leader master's
8a75602fc7db401e8fe48c4e4843e6f5 (127.31.21.193:39899): RUNNING [LEADER]
3cdc59582437453090162beef9e1e4f3 (127.31.21.194:43619): RUNNING
182e7dfef1ff4a2eb4e8619be80c4e65 (127.31.21.195:38585): RUNNING
All reported replicas are:
A = 8a75602fc7db401e8fe48c4e4843e6f5
B = 3cdc59582437453090162beef9e1e4f3
C = 182e7dfef1ff4a2eb4e8619be80c4e65
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 2 | -1 | Yes
B | A B C | 2 | -1 | Yes
C | A B C | 2 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+--------------------+---------------+---------+------------+------------------+-------------
pre_rebuild | 1 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20250809 14:08:38.024778 31831 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20250809 14:08:38.183656 5183 raft_consensus.cc:1273] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 2 FOLLOWER]: Refusing update from remote peer 8a75602fc7db401e8fe48c4e4843e6f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 214. Preceding OpId from leader: term: 2 index: 215. (index mismatch)
I20250809 14:08:38.184054 5419 consensus_queue.cc:1035] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 215, Last known committed idx: 214, Time since last communication: 0.000s
I20250809 14:08:38.185880 5317 raft_consensus.cc:1273] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 2 FOLLOWER]: Refusing update from remote peer 8a75602fc7db401e8fe48c4e4843e6f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 214. Preceding OpId from leader: term: 2 index: 215. (index mismatch)
I20250809 14:08:38.186139 5419 consensus_queue.cc:1035] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 215, Last known committed idx: 214, Time since last communication: 0.000s
I20250809 14:08:38.189813 5048 consensus_queue.cc:237] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 215, Committed index: 215, Last appended: 2.215, Last appended by leader: 214, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 216 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } }
I20250809 14:08:38.190313 5183 raft_consensus.cc:1273] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 2 FOLLOWER]: Refusing update from remote peer 8a75602fc7db401e8fe48c4e4843e6f5: Log matching property violated. Preceding OpId in replica: term: 2 index: 215. Preceding OpId from leader: term: 2 index: 216. (index mismatch)
I20250809 14:08:38.190537 5419 consensus_queue.cc:1035] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 216, Last known committed idx: 215, Time since last communication: 0.000s
I20250809 14:08:38.191197 5427 raft_consensus.cc:2953] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 2 LEADER]: Committing config change with OpId 2.216: config changed from index -1 to 216, VOTER 182e7dfef1ff4a2eb4e8619be80c4e65 (127.31.21.195) evicted. New config: { opid_index: 216 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } }
I20250809 14:08:38.191308 5183 raft_consensus.cc:2953] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 2 FOLLOWER]: Committing config change with OpId 2.216: config changed from index -1 to 216, VOTER 182e7dfef1ff4a2eb4e8619be80c4e65 (127.31.21.195) evicted. New config: { opid_index: 216 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } }
I20250809 14:08:38.192395 4894 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet f09f57826d7f4bb2be4fd10ea94a2a38 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250809 14:08:38.192543 4906 catalog_manager.cc:5582] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 reported cstate change: config changed from index -1 to 216, VOTER 182e7dfef1ff4a2eb4e8619be80c4e65 (127.31.21.195) evicted. New cstate: current_term: 2 leader_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" committed_config { opid_index: 216 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } }
I20250809 14:08:38.193814 5048 consensus_queue.cc:237] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 216, Committed index: 216, Last appended: 2.216, Last appended by leader: 214, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 217 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } }
I20250809 14:08:38.194073 5427 raft_consensus.cc:2953] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 2 LEADER]: Committing config change with OpId 2.217: config changed from index 216 to 217, VOTER 3cdc59582437453090162beef9e1e4f3 (127.31.21.194) evicted. New config: { opid_index: 217 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } }
I20250809 14:08:38.195021 4894 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet f09f57826d7f4bb2be4fd10ea94a2a38 with cas_config_opid_index 216: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250809 14:08:38.195348 4906 catalog_manager.cc:5582] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 reported cstate change: config changed from index 216 to 217, VOTER 3cdc59582437453090162beef9e1e4f3 (127.31.21.194) evicted. New cstate: current_term: 2 leader_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" committed_config { opid_index: 217 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } health_report { overall_health: HEALTHY } } }
I20250809 14:08:38.197023 5297 tablet_service.cc:1515] Processing DeleteTablet for tablet f09f57826d7f4bb2be4fd10ea94a2a38 with delete_type TABLET_DATA_TOMBSTONED (TS 182e7dfef1ff4a2eb4e8619be80c4e65 not found in new config with opid_index 216) from {username='slave'} at 127.0.0.1:58972
I20250809 14:08:38.197291 5445 tablet_replica.cc:331] stopping tablet replica
I20250809 14:08:38.197423 5445 raft_consensus.cc:2241] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250809 14:08:38.197487 5445 raft_consensus.cc:2270] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250809 14:08:38.197971 5445 ts_tablet_manager.cc:1905] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250809 14:08:38.198662 5163 tablet_service.cc:1515] Processing DeleteTablet for tablet f09f57826d7f4bb2be4fd10ea94a2a38 with delete_type TABLET_DATA_TOMBSTONED (TS 3cdc59582437453090162beef9e1e4f3 not found in new config with opid_index 217) from {username='slave'} at 127.0.0.1:57262
I20250809 14:08:38.198912 5448 tablet_replica.cc:331] stopping tablet replica
I20250809 14:08:38.199052 5448 raft_consensus.cc:2241] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250809 14:08:38.199177 5448 raft_consensus.cc:2270] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250809 14:08:38.199608 5445 ts_tablet_manager.cc:1918] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.215
I20250809 14:08:38.199616 5448 ts_tablet_manager.cc:1905] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250809 14:08:38.199749 5445 log.cc:1199] T f09f57826d7f4bb2be4fd10ea94a2a38 P 182e7dfef1ff4a2eb4e8619be80c4e65: Deleting WAL directory at /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/wal/wals/f09f57826d7f4bb2be4fd10ea94a2a38
I20250809 14:08:38.200348 4892 catalog_manager.cc:4928] TS 182e7dfef1ff4a2eb4e8619be80c4e65 (127.31.21.195:38585): tablet f09f57826d7f4bb2be4fd10ea94a2a38 (table pre_rebuild [id=c9f1c34f4ce247a6a8fa52596e6d514d]) successfully deleted
I20250809 14:08:38.201788 5448 ts_tablet_manager.cc:1918] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.216
I20250809 14:08:38.201860 5448 log.cc:1199] T f09f57826d7f4bb2be4fd10ea94a2a38 P 3cdc59582437453090162beef9e1e4f3: Deleting WAL directory at /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/wal/wals/f09f57826d7f4bb2be4fd10ea94a2a38
I20250809 14:08:38.202432 4891 catalog_manager.cc:4928] TS 3cdc59582437453090162beef9e1e4f3 (127.31.21.194:43619): tablet f09f57826d7f4bb2be4fd10ea94a2a38 (table pre_rebuild [id=c9f1c34f4ce247a6a8fa52596e6d514d]) successfully deleted
I20250809 14:08:38.203543 5028 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250809 14:08:38.203589 5297 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:38.205399 5163 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
da6e2529de77434fbe5d1474ba8eb8c3 | 127.31.21.254:42875 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.31.21.212:35329 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
182e7dfef1ff4a2eb4e8619be80c4e65 | 127.31.21.195:38585 | HEALTHY | <none> | 0 | 0
3cdc59582437453090162beef9e1e4f3 | 127.31.21.194:43619 | HEALTHY | <none> | 0 | 0
8a75602fc7db401e8fe48c4e4843e6f5 | 127.31.21.193:39899 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.31.21.193 | experimental | 127.31.21.193:39899
local_ip_for_outbound_sockets | 127.31.21.194 | experimental | 127.31.21.194:43619
local_ip_for_outbound_sockets | 127.31.21.195 | experimental | 127.31.21.195:38585
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data/info.pb | hidden | 127.31.21.193:39899
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data/info.pb | hidden | 127.31.21.194:43619
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data/info.pb | hidden | 127.31.21.195:38585
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.31.21.212:35329 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-------------+----+---------+---------------+---------+------------+------------------+-------------
pre_rebuild | 1 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 0
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 1
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20250809 14:08:38.213132 31831 log_verifier.cc:126] Checking tablet f09f57826d7f4bb2be4fd10ea94a2a38
I20250809 14:08:38.225754 31831 log_verifier.cc:177] Verified matching terms for 217 ops in tablet f09f57826d7f4bb2be4fd10ea94a2a38
I20250809 14:08:38.226230 4905 catalog_manager.cc:2482] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48782:
table { table_name: "pre_rebuild" } modify_external_catalogs: true
I20250809 14:08:38.226329 4905 catalog_manager.cc:2730] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48782:
table { table_name: "pre_rebuild" } modify_external_catalogs: true
I20250809 14:08:38.227064 4905 catalog_manager.cc:5869] T 00000000000000000000000000000000 P da6e2529de77434fbe5d1474ba8eb8c3: Sending DeleteTablet for 1 replicas of tablet f09f57826d7f4bb2be4fd10ea94a2a38
I20250809 14:08:38.227383 31831 test_util.cc:276] Using random seed: 974740917
I20250809 14:08:38.227375 5028 tablet_service.cc:1515] Processing DeleteTablet for tablet f09f57826d7f4bb2be4fd10ea94a2a38 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-08-09 14:08:38 UTC) from {username='slave'} at 127.0.0.1:46006
I20250809 14:08:38.227619 5452 tablet_replica.cc:331] stopping tablet replica
I20250809 14:08:38.227725 5452 raft_consensus.cc:2241] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 2 LEADER]: Raft consensus shutting down.
I20250809 14:08:38.227790 5452 raft_consensus.cc:2270] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250809 14:08:38.229094 5452 ts_tablet_manager.cc:1905] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250809 14:08:38.231161 5452 ts_tablet_manager.cc:1918] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.217
I20250809 14:08:38.231220 5452 log.cc:1199] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Deleting WAL directory at /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/wal/wals/f09f57826d7f4bb2be4fd10ea94a2a38
I20250809 14:08:38.231513 5452 ts_tablet_manager.cc:1939] T f09f57826d7f4bb2be4fd10ea94a2a38 P 8a75602fc7db401e8fe48c4e4843e6f5: Deleting consensus metadata
I20250809 14:08:38.231990 4894 catalog_manager.cc:4928] TS 8a75602fc7db401e8fe48c4e4843e6f5 (127.31.21.193:39899): tablet f09f57826d7f4bb2be4fd10ea94a2a38 (table pre_rebuild [id=c9f1c34f4ce247a6a8fa52596e6d514d]) successfully deleted
I20250809 14:08:38.232509 4905 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48842:
name: "post_rebuild"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250809 14:08:38.232785 4905 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table post_rebuild in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250809 14:08:38.234256 5028 tablet_service.cc:1468] Processing CreateTablet for tablet aeba1747c9a443d3b995e801cd37c1fc (DEFAULT_TABLE table=post_rebuild [id=c5595615b9cb47e186f3e843b8a55dd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:38.234316 5163 tablet_service.cc:1468] Processing CreateTablet for tablet aeba1747c9a443d3b995e801cd37c1fc (DEFAULT_TABLE table=post_rebuild [id=c5595615b9cb47e186f3e843b8a55dd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:38.234409 5028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeba1747c9a443d3b995e801cd37c1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:38.234498 5163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeba1747c9a443d3b995e801cd37c1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:38.235409 5458 tablet_bootstrap.cc:492] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5: Bootstrap starting.
I20250809 14:08:38.235500 5297 tablet_service.cc:1468] Processing CreateTablet for tablet aeba1747c9a443d3b995e801cd37c1fc (DEFAULT_TABLE table=post_rebuild [id=c5595615b9cb47e186f3e843b8a55dd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250809 14:08:38.235662 5297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeba1747c9a443d3b995e801cd37c1fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250809 14:08:38.235791 5458 tablet_bootstrap.cc:654] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:38.236235 5458 tablet_bootstrap.cc:492] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5: No bootstrap required, opened a new log
I20250809 14:08:38.236274 5458 ts_tablet_manager.cc:1397] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:38.236383 5458 raft_consensus.cc:357] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:38.236438 5458 raft_consensus.cc:383] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:38.236459 5458 raft_consensus.cc:738] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a75602fc7db401e8fe48c4e4843e6f5, State: Initialized, Role: FOLLOWER
I20250809 14:08:38.236514 5458 consensus_queue.cc:260] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:38.236596 5458 ts_tablet_manager.cc:1428] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:38.237076 5461 tablet_bootstrap.cc:492] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65: Bootstrap starting.
I20250809 14:08:38.237226 5459 tablet_bootstrap.cc:492] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3: Bootstrap starting.
I20250809 14:08:38.237377 5461 tablet_bootstrap.cc:654] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:38.237571 5459 tablet_bootstrap.cc:654] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3: Neither blocks nor log segments found. Creating new log.
I20250809 14:08:38.237934 5461 tablet_bootstrap.cc:492] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65: No bootstrap required, opened a new log
I20250809 14:08:38.237977 5461 ts_tablet_manager.cc:1397] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:38.238095 5459 tablet_bootstrap.cc:492] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3: No bootstrap required, opened a new log
I20250809 14:08:38.238106 5461 raft_consensus.cc:357] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:38.238143 5459 ts_tablet_manager.cc:1397] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20250809 14:08:38.238162 5461 raft_consensus.cc:383] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:38.238205 5461 raft_consensus.cc:738] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 182e7dfef1ff4a2eb4e8619be80c4e65, State: Initialized, Role: FOLLOWER
I20250809 14:08:38.238253 5461 consensus_queue.cc:260] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:38.238261 5459 raft_consensus.cc:357] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:38.238315 5459 raft_consensus.cc:383] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250809 14:08:38.238346 5461 ts_tablet_manager.cc:1428] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:38.238391 5459 raft_consensus.cc:738] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cdc59582437453090162beef9e1e4f3, State: Initialized, Role: FOLLOWER
I20250809 14:08:38.238464 5459 consensus_queue.cc:260] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:38.238579 5459 ts_tablet_manager.cc:1428] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20250809 14:08:38.275458 5430 raft_consensus.cc:491] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250809 14:08:38.275571 5430 raft_consensus.cc:513] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:38.275903 5430 leader_election.cc:290] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a75602fc7db401e8fe48c4e4843e6f5 (127.31.21.193:39899), 3cdc59582437453090162beef9e1e4f3 (127.31.21.194:43619)
I20250809 14:08:38.278517 5183 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "aeba1747c9a443d3b995e801cd37c1fc" candidate_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cdc59582437453090162beef9e1e4f3" is_pre_election: true
I20250809 14:08:38.278517 5048 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "aeba1747c9a443d3b995e801cd37c1fc" candidate_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" is_pre_election: true
I20250809 14:08:38.278607 5183 raft_consensus.cc:2466] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182e7dfef1ff4a2eb4e8619be80c4e65 in term 0.
I20250809 14:08:38.278635 5048 raft_consensus.cc:2466] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 182e7dfef1ff4a2eb4e8619be80c4e65 in term 0.
I20250809 14:08:38.278786 5249 leader_election.cc:304] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182e7dfef1ff4a2eb4e8619be80c4e65, 3cdc59582437453090162beef9e1e4f3; no voters:
I20250809 14:08:38.278930 5430 raft_consensus.cc:2802] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250809 14:08:38.278990 5430 raft_consensus.cc:491] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250809 14:08:38.279019 5430 raft_consensus.cc:3058] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:38.279732 5430 raft_consensus.cc:513] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:38.279883 5430 leader_election.cc:290] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [CANDIDATE]: Term 1 election: Requested vote from peers 8a75602fc7db401e8fe48c4e4843e6f5 (127.31.21.193:39899), 3cdc59582437453090162beef9e1e4f3 (127.31.21.194:43619)
I20250809 14:08:38.280006 5048 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "aeba1747c9a443d3b995e801cd37c1fc" candidate_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a75602fc7db401e8fe48c4e4843e6f5"
I20250809 14:08:38.280021 5183 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "aeba1747c9a443d3b995e801cd37c1fc" candidate_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cdc59582437453090162beef9e1e4f3"
I20250809 14:08:38.280085 5048 raft_consensus.cc:3058] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:38.280088 5183 raft_consensus.cc:3058] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3 [term 0 FOLLOWER]: Advancing to term 1
I20250809 14:08:38.280709 5183 raft_consensus.cc:2466] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182e7dfef1ff4a2eb4e8619be80c4e65 in term 1.
I20250809 14:08:38.280886 5048 raft_consensus.cc:2466] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 182e7dfef1ff4a2eb4e8619be80c4e65 in term 1.
I20250809 14:08:38.280889 5249 leader_election.cc:304] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 182e7dfef1ff4a2eb4e8619be80c4e65, 3cdc59582437453090162beef9e1e4f3; no voters:
I20250809 14:08:38.280972 5430 raft_consensus.cc:2802] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 1 FOLLOWER]: Leader election won for term 1
I20250809 14:08:38.281140 5430 raft_consensus.cc:695] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [term 1 LEADER]: Becoming Leader. State: Replica: 182e7dfef1ff4a2eb4e8619be80c4e65, State: Running, Role: LEADER
I20250809 14:08:38.281222 5430 consensus_queue.cc:237] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } }
I20250809 14:08:38.281841 4905 catalog_manager.cc:5582] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 reported cstate change: term changed from 0 to 1, leader changed from <none> to 182e7dfef1ff4a2eb4e8619be80c4e65 (127.31.21.195). New cstate: current_term: 1 leader_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "182e7dfef1ff4a2eb4e8619be80c4e65" member_type: VOTER last_known_addr { host: "127.31.21.195" port: 38585 } health_report { overall_health: HEALTHY } } }
I20250809 14:08:38.303637 5048 raft_consensus.cc:1273] T aeba1747c9a443d3b995e801cd37c1fc P 8a75602fc7db401e8fe48c4e4843e6f5 [term 1 FOLLOWER]: Refusing update from remote peer 182e7dfef1ff4a2eb4e8619be80c4e65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20250809 14:08:38.303699 5183 raft_consensus.cc:1273] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3 [term 1 FOLLOWER]: Refusing update from remote peer 182e7dfef1ff4a2eb4e8619be80c4e65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250809 14:08:38.303719 5230 tablet.cc:2378] T aeba1747c9a443d3b995e801cd37c1fc P 3cdc59582437453090162beef9e1e4f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250809 14:08:38.303982 5473 consensus_queue.cc:1035] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cdc59582437453090162beef9e1e4f3" member_type: VOTER last_known_addr { host: "127.31.21.194" port: 43619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250809 14:08:38.304121 5466 consensus_queue.cc:1035] T aeba1747c9a443d3b995e801cd37c1fc P 182e7dfef1ff4a2eb4e8619be80c4e65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a75602fc7db401e8fe48c4e4843e6f5" member_type: VOTER last_known_addr { host: "127.31.21.193" port: 39899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250809 14:08:38.305876 5432 mvcc.cc:204] Tried to move back new op lower bound from 7187449930969239552 to 7187449930880339968. Current Snapshot: MvccSnapshot[applied={T|T < 7187449930969239552}]
I20250809 14:08:38.306377 5429 mvcc.cc:204] Tried to move back new op lower bound from 7187449930971275264 to 7187449930880339968. Current Snapshot: MvccSnapshot[applied={T|T < 7187449930969239552}]
I20250809 14:08:38.306799 5428 mvcc.cc:204] Tried to move back new op lower bound from 7187449930971275264 to 7187449930880339968. Current Snapshot: MvccSnapshot[applied={T|T < 7187449930969239552}]
I20250809 14:08:38.468991 5028 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:38.469038 5163 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250809 14:08:38.469151 5297 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+---------------------+---------
da6e2529de77434fbe5d1474ba8eb8c3 | 127.31.21.254:42875 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_ca_key_size | 768 | experimental | all 1 server(s) checked
ipki_server_key_size | 768 | experimental | all 1 server(s) checked
never_fsync | true | unsafe,advanced | all 1 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 1 server(s) checked
rpc_reuseport | true | experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
server_dump_info_format | pb | hidden | all 1 server(s) checked
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/master-0/data/info.pb | hidden | all 1 server(s) checked
tsk_num_rsa_bits | 512 | experimental | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.31.21.212:35329 | all 1 server(s) checked
time_source | builtin | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
182e7dfef1ff4a2eb4e8619be80c4e65 | 127.31.21.195:38585 | HEALTHY | <none> | 1 | 0
3cdc59582437453090162beef9e1e4f3 | 127.31.21.194:43619 | HEALTHY | <none> | 0 | 0
8a75602fc7db401e8fe48c4e4843e6f5 | 127.31.21.193:39899 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
ipki_server_key_size | 768 | experimental | all 3 server(s) checked
local_ip_for_outbound_sockets | 127.31.21.193 | experimental | 127.31.21.193:39899
local_ip_for_outbound_sockets | 127.31.21.194 | experimental | 127.31.21.194:43619
local_ip_for_outbound_sockets | 127.31.21.195 | experimental | 127.31.21.195:38585
never_fsync | true | unsafe,advanced | all 3 server(s) checked
openssl_security_level_override | 0 | unsafe,hidden | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
server_dump_info_format | pb | hidden | all 3 server(s) checked
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-0/data/info.pb | hidden | 127.31.21.193:39899
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-1/data/info.pb | hidden | 127.31.21.194:43619
server_dump_info_path | /tmp/dist-test-taskwLxUSP/test-tmp/kudu-admin-test.5.IsSecure_SecureClusterAdminCliParamTest.TestRebuildMaster_0.1754748496992630-31831-0/minicluster-data/ts-2/data/info.pb | hidden | 127.31.21.195:38585
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+---------------------+-------------------------
builtin_ntp_servers | 127.31.21.212:35329 | all 3 server(s) checked
time_source | builtin | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
--------------+----+---------+---------------+---------+------------+------------------+-------------
post_rebuild | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20250809 14:08:38.479477 31831 log_verifier.cc:126] Checking tablet aeba1747c9a443d3b995e801cd37c1fc
I20250809 14:08:38.513777 31831 log_verifier.cc:177] Verified matching terms for 206 ops in tablet aeba1747c9a443d3b995e801cd37c1fc
I20250809 14:08:38.513873 31831 log_verifier.cc:126] Checking tablet f09f57826d7f4bb2be4fd10ea94a2a38
I20250809 14:08:38.513964 31831 log_verifier.cc:177] Verified matching terms for 0 ops in tablet f09f57826d7f4bb2be4fd10ea94a2a38
I20250809 14:08:38.515609 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 4946
I20250809 14:08:38.521865 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 5099
I20250809 14:08:38.526549 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 5233
I20250809 14:08:38.531163 31831 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskwLxUSP/build/release/bin/kudu with pid 4876
2025-08-09T14:08:38Z chronyd exiting
[ OK ] IsSecure/SecureClusterAdminCliParamTest.TestRebuildMaster/0 (3670 ms)
[----------] 1 test from IsSecure/SecureClusterAdminCliParamTest (3670 ms total)
[----------] Global test environment tear-down
[==========] 9 tests from 5 test suites ran. (21557 ms total)
[ PASSED ] 8 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] AdminCliTest.TestRebuildTables
1 FAILED TEST