07:27:26.238 [INFO - main] (KuduTestHarness.java:155) Creating a new MiniKuduCluster...
07:27:26.238 [INFO - main] (TempDirUtils.java:71) Using the temp directory defined by TEST_TMPDIR: /tmp/dist-test-taskO_PLPU/test-tmp
07:27:26.238 [INFO - main] (KuduBinaryLocator.java:83) Using Kudu binary directory specified by system property 'kuduBinDir': ../../../build/tsan/bin
07:27:26.239 [INFO - main] (MiniKuduCluster.java:217) Starting process: [/tmp/dist-test-taskO_PLPU/java/build/dist-test/../../../build/tsan/bin/kudu, test, mini_cluster, --serialization=pb]
07:27:26.240 [DEBUG - main] (MiniKuduCluster.java:185) Request: create_cluster {
num_masters: 3
num_tservers: 3
enable_kerberos: false
cluster_root: "/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007"
hms_mode: NONE
mini_kdc_options {
}
principal: "kudu"
mini_oidc_options {
}
}
07:27:26.612 [DEBUG - main] (MiniKuduCluster.java:195) Response:
07:27:26.612 [DEBUG - main] (MiniKuduCluster.java:185) Request: start_cluster {
}
07:27:26.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:26.615957 5329 mini_oidc.cc:152] Starting JWKS server
07:27:26.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:26.620213 5329 webserver.cc:460] Webserver started at http://127.0.0.1:44987/ using document root <none> and password file <none>
07:27:26.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:26.620783 5329 mini_oidc.cc:189] Starting OIDC Discovery server
07:27:26.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:26.622888 5329 webserver.cc:460] Webserver started at http://127.0.0.1:44519/ using document root <none> and password file <none>
07:27:26.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-07T07:27:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
07:27:26.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-07T07:27:26Z Disabled control of system clock
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:26.682104 5329 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/wal
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/data
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/logs
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/data/info.pb
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:27:26.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.5.52.126:33565
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.5.52.126
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.5.52.84:41079
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.5.52.126:33565,127.5.52.125:45789,127.5.52.124:38815
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:26.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:44987/jwks.json with env {}
07:27:27.006 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:27.005656 5347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:27:27.006 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:27.006321 5347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:27:27.007 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:27.006826 5347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:27:27.040 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:27.040203 5347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:27:27.040 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:27.040500 5347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:27:27.040 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:27.040697 5347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:27:27.041 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:27.040891 5347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:27:27.041 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:27.041091 5347 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:27:27.041 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:27.041237 5347 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:44987/jwks.json
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:27.080775 5347 master_runner.cc:386] Master server non-default flags:
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.5.52.84:41079
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/data
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/wal
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.5.52.126:33565,127.5.52.125:45789,127.5.52.124:38815
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.5.52.126:33565
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:44987/jwks.json
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/data/info.pb
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.5.52.126
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/logs
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:27:27.081 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:27:27.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 07 Aug 2026 06:41:47 UTC on ccycloud.quasar-byjrmo.root.comops.site
07:27:27.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4571901
07:27:27.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:27:27.082 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:27.082115 5347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:27:27.084 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:27.084331 5347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:27:27.104 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:27.104350 5354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:28.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:27.104387 5356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:28.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:27.105863 5347 server_base.cc:1034] running on GCE node
07:27:28.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:27.104542 5359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:28.342 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.342090 5347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:27:28.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.345381 5347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:27:28.347 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.346813 5347 hybrid_clock.cc:648] HybridClock initialized: now 1786087648346821 us; error 100 us; skew 500 ppm
07:27:28.347 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.347635 5347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:27:28.358 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.358398 5347 webserver.cc:460] Webserver started at http://127.5.52.126:43957/ using document root <none> and password file <none>
07:27:28.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.359375 5347 fs_manager.cc:359] Metadata directory not provided
07:27:28.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.359575 5347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:27:28.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.359998 5347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:27:28.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.365114 5347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/data/instance:
07:27:28.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "19dd34855bc94452b795a818eb8d9212"
07:27:28.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:28 on dist-test-slave-7nm7"
07:27:28.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.366214 5347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/wal/instance:
07:27:28.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "19dd34855bc94452b795a818eb8d9212"
07:27:28.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:28 on dist-test-slave-7nm7"
07:27:28.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.374492 5347 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.005s
07:27:28.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.380326 5375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:27:28.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.381549 5347 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.381857 5347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/data,/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/wal
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "19dd34855bc94452b795a818eb8d9212"
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:28 on dist-test-slave-7nm7"
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.382169 5347 fs_report.cc:383] FS layout report
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/wal
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/wal
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/data/data
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:27:28.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:27:28.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:27:28.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:27:28.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:27:28.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:27:28.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:27:28.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:27:28.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.430780 5347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:27:28.432 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.432281 5347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:27:28.433 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.432801 5347 kserver.cc:163] Server-wide thread pool size limit: 3276
07:27:28.522 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.522481 5347 rpc_server.cc:307] RPC server started. Bound to: 127.5.52.126:33565
07:27:28.522 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.522598 5442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.52.126:33565 every 8 connection(s)
07:27:28.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.525327 5347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/data/info.pb
07:27:28.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.531592 5443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
07:27:28.535 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.535604 5329 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu as pid 5347
07:27:28.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.536028 5329 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-0/wal/instance
07:27:28.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.539523 5329 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:28.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:28.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/wal
07:27:28.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/data
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/logs
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/data/info.pb
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.5.52.125:45789
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.5.52.125
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.5.52.84:41079
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.5.52.126:33565,127.5.52.125:45789,127.5.52.124:38815
07:27:28.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:28.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:44987/jwks.json with env {}
07:27:28.547 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.545042 5443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } has no permanent_uuid. Determining permanent_uuid...
07:27:28.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.576504 5443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } has no permanent_uuid. Determining permanent_uuid...
07:27:28.579 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.578984 5376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.52.125:45789: connect: Connection refused (error 111)
07:27:28.583 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.583218 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.125:45789: Network error: Client connection negotiation failed: client connection to 127.5.52.125:45789: connect: Connection refused (error 111)
07:27:28.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.632808 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } attempt: 1
07:27:28.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.636950 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.125:45789: Network error: Client connection negotiation failed: client connection to 127.5.52.125:45789: connect: Connection refused (error 111)
07:27:28.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.705536 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } attempt: 2
07:27:28.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.709967 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.125:45789: Network error: Client connection negotiation failed: client connection to 127.5.52.125:45789: connect: Connection refused (error 111)
07:27:28.801 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.801452 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } attempt: 3
07:27:28.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.805769 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.125:45789: Network error: Client connection negotiation failed: client connection to 127.5.52.125:45789: connect: Connection refused (error 111)
07:27:28.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.884411 5446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:27:28.885 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.885020 5446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:27:28.885 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.885519 5446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:27:28.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.919072 5446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:27:28.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.919370 5446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:27:28.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.919561 5446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:27:28.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.919761 5446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:27:28.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.919950 5446 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:27:28.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.920090 5446 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:44987/jwks.json
07:27:28.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.949239 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } attempt: 4
07:27:28.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.953569 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.125:45789: Network error: Client connection negotiation failed: client connection to 127.5.52.125:45789: connect: Connection refused (error 111)
07:27:28.960 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.960218 5446 master_runner.cc:386] Master server non-default flags:
07:27:28.960 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.5.52.84:41079
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/data
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/wal
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.5.52.126:33565,127.5.52.125:45789,127.5.52.124:38815
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.5.52.125:45789
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:44987/jwks.json
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/data/info.pb
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.5.52.125
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/logs
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:27:28.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:27:28.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 07 Aug 2026 06:41:47 UTC on ccycloud.quasar-byjrmo.root.comops.site
07:27:28.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4571901
07:27:28.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:27:28.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.961402 5446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:27:28.963 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:28.963140 5446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:27:28.984 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.983384 5466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:29.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:29.253036 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } attempt: 5
07:27:29.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:29.257690 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.125:45789: Network error: Client connection negotiation failed: client connection to 127.5.52.125:45789: connect: Connection refused (error 111)
07:27:29.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:29.805526 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } attempt: 6
07:27:29.814 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:29.814335 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.125:45789: Network error: Client connection negotiation failed: client connection to 127.5.52.125:45789: connect: Connection refused (error 111)
07:27:30.216 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.983665 5465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:30.216 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:28.983831 5470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:30.217 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.217000 5468 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
07:27:30.217 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.217355 5446 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
07:27:30.221 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.221650 5446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:27:30.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.224408 5446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:27:30.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.225768 5446 hybrid_clock.cc:648] HybridClock initialized: now 1786087650225729 us; error 48 us; skew 500 ppm
07:27:30.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.226739 5446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:27:30.233 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.233186 5446 webserver.cc:460] Webserver started at http://127.5.52.125:34933/ using document root <none> and password file <none>
07:27:30.234 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.234071 5446 fs_manager.cc:359] Metadata directory not provided
07:27:30.234 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.234423 5446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:27:30.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.234977 5446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:27:30.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.239964 5446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/data/instance:
07:27:30.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "c7964e5424a54cf3a50405206fbc48bd"
07:27:30.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:30 on dist-test-slave-7nm7"
07:27:30.241 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.241051 5446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/wal/instance:
07:27:30.241 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "c7964e5424a54cf3a50405206fbc48bd"
07:27:30.241 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:30 on dist-test-slave-7nm7"
07:27:30.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.248769 5446 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
07:27:30.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.255297 5487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:27:30.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.256620 5446 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
07:27:30.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.256928 5446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/data,/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/wal
07:27:30.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "c7964e5424a54cf3a50405206fbc48bd"
07:27:30.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:30 on dist-test-slave-7nm7"
07:27:30.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.257266 5446 fs_report.cc:383] FS layout report
07:27:30.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:27:30.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/wal
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/wal
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/data/data
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:27:30.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:27:30.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.302886 5446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:27:30.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.304528 5446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:27:30.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.304996 5446 kserver.cc:163] Server-wide thread pool size limit: 3276
07:27:30.394 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.394064 5446 rpc_server.cc:307] RPC server started. Bound to: 127.5.52.125:45789
07:27:30.394 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.394415 5574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.52.125:45789 every 8 connection(s)
07:27:30.397 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.397058 5446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/data/info.pb
07:27:30.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.398813 5329 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu as pid 5446
07:27:30.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.399257 5329 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-1/wal/instance
07:27:30.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.402765 5329 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:30.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:30.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/wal
07:27:30.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/data
07:27:30.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:27:30.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:27:30.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:27:30.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:30.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:30.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:30.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/logs
07:27:30.403 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/data/info.pb
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.5.52.124:38815
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.5.52.124
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.5.52.84:41079
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.5.52.126:33565,127.5.52.125:45789,127.5.52.124:38815
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:30.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:44987/jwks.json with env {}
07:27:30.407 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.405148 5575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
07:27:30.420 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.418623 5575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } has no permanent_uuid. Determining permanent_uuid...
07:27:30.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.452732 5575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } has no permanent_uuid. Determining permanent_uuid...
07:27:30.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.471571 5575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } has no permanent_uuid. Determining permanent_uuid...
07:27:30.474 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.473875 5494 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:30.478 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.478540 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:30.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.528214 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 1
07:27:30.533 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.533202 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:30.602 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.601660 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 2
07:27:30.606 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.606145 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:30.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.697589 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 3
07:27:30.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.701627 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:30.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.757328 5579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:27:30.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.758426 5579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:27:30.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.759061 5579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:27:30.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.793334 5579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:27:30.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.793743 5579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:27:30.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.794076 5579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:27:30.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.794397 5579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:27:30.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.794713 5579 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:27:30.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.794948 5579 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:44987/jwks.json
07:27:30.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.837247 5579 master_runner.cc:386] Master server non-default flags:
07:27:30.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:30.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.5.52.84:41079
07:27:30.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:30.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:30.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/data
07:27:30.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/wal
07:27:30.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:30.837 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.5.52.126:33565,127.5.52.125:45789,127.5.52.124:38815
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.5.52.124:38815
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:44987/jwks.json
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/data/info.pb
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.5.52.124
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/logs
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 07 Aug 2026 06:41:47 UTC on ccycloud.quasar-byjrmo.root.comops.site
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4571901
07:27:30.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:27:30.839 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.838900 5579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:27:30.840 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.840612 5579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:27:30.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.845089 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 4
07:27:30.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.850020 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:30.860 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.860322 5594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:30.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.875017 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } attempt: 7
07:27:30.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.889930 5443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } has no permanent_uuid. Determining permanent_uuid...
07:27:30.897 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.896615 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:30.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.955178 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 1
07:27:30.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.960963 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:31.043 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:31.042577 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 2
07:27:31.046 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:31.046478 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:31.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:31.131877 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 3
07:27:31.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:31.136950 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:31.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:31.149647 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 5
07:27:31.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:31.154381 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:31.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:31.277410 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 4
07:27:31.283 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:31.283054 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:31.567 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:31.566529 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 5
07:27:31.572 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:31.572422 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:31.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:31.701792 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 6
07:27:31.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:31.708961 5575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:32.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.124866 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 6
07:27:32.131 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:32.131366 5443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.52.124:38815: Network error: Client connection negotiation failed: client connection to 127.5.52.124:38815: connect: Connection refused (error 111)
07:27:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.860747 5593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:30.861852 5598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:32.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:30.864471 5579 server_base.cc:1034] running on GCE node
07:27:32.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.158442 5579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:27:32.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.161180 5579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:27:32.162 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.162609 5579 hybrid_clock.cc:648] HybridClock initialized: now 1786087652162556 us; error 60 us; skew 500 ppm
07:27:32.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.163532 5579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:27:32.170 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.170542 5579 webserver.cc:460] Webserver started at http://127.5.52.124:41541/ using document root <none> and password file <none>
07:27:32.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.171530 5579 fs_manager.cc:359] Metadata directory not provided
07:27:32.172 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.171788 5579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:27:32.172 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.172323 5579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:27:32.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.177294 5579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/data/instance:
07:27:32.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "25ee7bac5dfe4bc58ca124c5def320b0"
07:27:32.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:32 on dist-test-slave-7nm7"
07:27:32.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.178457 5579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/wal/instance:
07:27:32.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "25ee7bac5dfe4bc58ca124c5def320b0"
07:27:32.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:32 on dist-test-slave-7nm7"
07:27:32.186 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.186604 5579 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
07:27:32.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.192600 5617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:27:32.193 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.193681 5579 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
07:27:32.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.193962 5579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/data,/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/wal
07:27:32.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "25ee7bac5dfe4bc58ca124c5def320b0"
07:27:32.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:32 on dist-test-slave-7nm7"
07:27:32.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.194339 5579 fs_report.cc:383] FS layout report
07:27:32.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:27:32.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/wal
07:27:32.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/wal
07:27:32.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/data/data
07:27:32.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:27:32.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:27:32.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:27:32.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:27:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:27:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:27:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:27:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:27:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:27:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:27:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:27:32.244 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.244004 5579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:27:32.245 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.245507 5579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:27:32.246 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.245951 5579 kserver.cc:163] Server-wide thread pool size limit: 3276
07:27:32.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.331511 5579 rpc_server.cc:307] RPC server started. Bound to: 127.5.52.124:38815
07:27:32.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.331609 5692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.52.124:38815 every 8 connection(s)
07:27:32.335 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.334882 5579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/data/info.pb
07:27:32.336 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.335901 5329 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu as pid 5579
07:27:32.336 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.336323 5329 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/master-2/wal/instance
07:27:32.342 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.342235 5702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
07:27:32.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.366583 5702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } has no permanent_uuid. Determining permanent_uuid...
07:27:32.407 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.406992 5702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } has no permanent_uuid. Determining permanent_uuid...
07:27:32.423 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.423224 5702 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } has no permanent_uuid. Determining permanent_uuid...
07:27:32.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.443663 5702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0: Bootstrap starting.
07:27:32.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.450310 5702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0: Neither blocks nor log segments found. Creating new log.
07:27:32.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.452365 5702 log.cc:824] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0: Log is configured to *not* fsync() on all Append() calls
07:27:32.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.457528 5702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0: No bootstrap required, opened a new log
07:27:32.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.479399 5702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } }
07:27:32.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.480091 5702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
07:27:32.480 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.480384 5702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25ee7bac5dfe4bc58ca124c5def320b0, State: Initialized, Role: FOLLOWER
07:27:32.481 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.481072 5702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } }
07:27:32.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.484156 5716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } } }
07:27:32.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.485082 5716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [sys.catalog]: This master's current role is: FOLLOWER
07:27:32.486 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.486020 5702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [sys.catalog]: configured and running, proceeding with master startup.
07:27:32.502 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:32.501965 5626 tablet.cc:2369] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
07:27:32.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:32.515225 5736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
07:27:32.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:32.515578 5736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
07:27:32.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.510104 5579 master_runner.cc:186] Error getting master registration for 127.5.52.126:33565: OK, instance_id { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" instance_seqno: 1786087648497568 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
07:27:32.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.536707 5579 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
07:27:32.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.769706 5575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 7
07:27:32.791 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.791028 5575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd: Bootstrap starting.
07:27:32.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.796696 5575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd: Neither blocks nor log segments found. Creating new log.
07:27:32.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.798727 5575 log.cc:824] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd: Log is configured to *not* fsync() on all Append() calls
07:27:32.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.805743 5575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd: No bootstrap required, opened a new log
07:27:32.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.826807 5575 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } }
07:27:32.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.827531 5575 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
07:27:32.828 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.827771 5575 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7964e5424a54cf3a50405206fbc48bd, State: Initialized, Role: FOLLOWER
07:27:32.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.828457 5575 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } }
07:27:32.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.831202 5743 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } } }
07:27:32.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.832089 5743 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [sys.catalog]: This master's current role is: FOLLOWER
07:27:32.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.833115 5575 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [sys.catalog]: configured and running, proceeding with master startup.
07:27:32.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:32.875283 5757 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
07:27:32.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:32.875880 5757 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
07:27:32.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.890305 5743 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
07:27:32.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.890934 5743 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } }
07:27:32.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.897293 5655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c7964e5424a54cf3a50405206fbc48bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" is_pre_election: true
07:27:32.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.859251 5446 master_runner.cc:186] Error getting master registration for 127.5.52.126:33565: OK, instance_id { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" instance_seqno: 1786087648497568 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
07:27:32.899 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.898722 5446 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
07:27:32.899 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.898797 5655 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7964e5424a54cf3a50405206fbc48bd in term 0.
07:27:32.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.900048 5743 leader_election.cc:288] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19dd34855bc94452b795a818eb8d9212 (127.5.52.126:33565), 25ee7bac5dfe4bc58ca124c5def320b0 (127.5.52.124:38815)
07:27:32.901 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.900250 5494 leader_election.cc:302] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25ee7bac5dfe4bc58ca124c5def320b0, c7964e5424a54cf3a50405206fbc48bd; no voters:
07:27:32.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.909466 5743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [term 0 FOLLOWER]: Leader pre-election won for term 1
07:27:32.910 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.909876 5743 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
07:27:32.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.910966 5743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [term 0 FOLLOWER]: Advancing to term 1
07:27:32.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.918215 5743 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } }
07:27:32.927 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.921545 5743 leader_election.cc:288] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [CANDIDATE]: Term 1 election: Requested vote from peers 19dd34855bc94452b795a818eb8d9212 (127.5.52.126:33565), 25ee7bac5dfe4bc58ca124c5def320b0 (127.5.52.124:38815)
07:27:32.927 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.922371 5655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c7964e5424a54cf3a50405206fbc48bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25ee7bac5dfe4bc58ca124c5def320b0"
07:27:32.927 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.922957 5655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [term 0 FOLLOWER]: Advancing to term 1
07:27:32.930 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.930534 5655 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7964e5424a54cf3a50405206fbc48bd in term 1.
07:27:32.932 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.931718 5494 leader_election.cc:302] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25ee7bac5dfe4bc58ca124c5def320b0, c7964e5424a54cf3a50405206fbc48bd; no voters:
07:27:32.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.932737 5743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [term 1 FOLLOWER]: Leader election won for term 1
07:27:32.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.933881 5743 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [term 1 LEADER]: Becoming Leader. State: Replica: c7964e5424a54cf3a50405206fbc48bd, State: Running, Role: LEADER
07:27:32.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.935240 5743 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } }
07:27:32.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.944849 5759 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7964e5424a54cf3a50405206fbc48bd. Latest consensus state: current_term: 1 leader_uuid: "c7964e5424a54cf3a50405206fbc48bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } } }
07:27:32.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.945681 5759 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [sys.catalog]: This master's current role is: LEADER
07:27:32.953 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.953030 5762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
07:27:32.970 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:32.970083 5762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
07:27:33.005 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.004117 5655 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [term 1 FOLLOWER]: Refusing update from remote peer c7964e5424a54cf3a50405206fbc48bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
07:27:33.006 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.006209 5743 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
07:27:33.037 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.036809 5767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7964e5424a54cf3a50405206fbc48bd. Latest consensus state: current_term: 1 leader_uuid: "c7964e5424a54cf3a50405206fbc48bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } } }
07:27:33.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.037750 5767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [sys.catalog]: This master's current role is: FOLLOWER
07:27:33.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.048553 5768 mvcc.cc:205] Tried to move back new op lower bound from 7315815026682580992 to 7315815026434301952. Current Snapshot: MvccSnapshot[applied={T|T < 7315815026682580992}]
07:27:33.054 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.053535 5743 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7964e5424a54cf3a50405206fbc48bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } } }
07:27:33.054 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.054409 5743 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [sys.catalog]: This master's current role is: LEADER
07:27:33.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.059607 5767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7964e5424a54cf3a50405206fbc48bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } } }
07:27:33.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.060680 5767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0 [sys.catalog]: This master's current role is: FOLLOWER
07:27:33.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.062136 5762 catalog_manager.cc:1347] Generated new cluster ID: 62f0eb66c5714fbcbb050c988fc55bf5
07:27:33.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.062713 5762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
07:27:33.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.119313 5762 catalog_manager.cc:1370] Generated new certificate authority record
07:27:33.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.125317 5762 catalog_manager.cc:1504] Loading token signing keys...
07:27:33.163 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.162039 5762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd: Generated new TSK 0
07:27:33.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.164929 5443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } attempt: 7
07:27:33.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.165726 5762 catalog_manager.cc:1514] Initializing in-progress tserver states...
07:27:33.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.205678 5329 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:33.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:33.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/wal
07:27:33.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/data
07:27:33.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:27:33.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:27:33.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/logs
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/data/info.pb
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.5.52.65:0
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.5.52.65
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.5.52.65
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.5.52.126:33565,127.5.52.125:45789,127.5.52.124:38815
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.5.52.84:41079
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:33.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
07:27:33.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.222492 5443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212: Bootstrap starting.
07:27:33.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.239710 5443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212: Neither blocks nor log segments found. Creating new log.
07:27:33.245 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.245155 5443 log.cc:824] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212: Log is configured to *not* fsync() on all Append() calls
07:27:33.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.273763 5443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212: No bootstrap required, opened a new log
07:27:33.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.374356 5443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } }
07:27:33.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.376834 5443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
07:27:33.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.377772 5443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19dd34855bc94452b795a818eb8d9212, State: Initialized, Role: FOLLOWER
07:27:33.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.380851 5443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } }
07:27:33.407 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.395424 5778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } } }
07:27:33.408 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.407042 5778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [sys.catalog]: This master's current role is: FOLLOWER
07:27:33.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.420939 5408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c7964e5424a54cf3a50405206fbc48bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19dd34855bc94452b795a818eb8d9212"
07:27:33.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.421797 5408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [term 0 FOLLOWER]: Advancing to term 1
07:27:33.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.426772 5409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c7964e5424a54cf3a50405206fbc48bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19dd34855bc94452b795a818eb8d9212" is_pre_election: true
07:27:33.435 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.434836 5443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [sys.catalog]: configured and running, proceeding with master startup.
07:27:33.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:33.443467 5384 tablet.cc:2369] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212: Can't schedule compaction. Clean time has not been advanced past its initial value.
07:27:33.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.444427 5408 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7964e5424a54cf3a50405206fbc48bd in term 1.
07:27:33.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.445830 5407 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [term 1 FOLLOWER]: Refusing update from remote peer c7964e5424a54cf3a50405206fbc48bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
07:27:33.454 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.453084 5759 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
07:27:33.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.527966 5736 catalog_manager.cc:1259] Loaded cluster ID: 62f0eb66c5714fbcbb050c988fc55bf5
07:27:33.533 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.533254 5736 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0: loading cluster ID for follower catalog manager: success
07:27:33.550 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.549986 5778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7964e5424a54cf3a50405206fbc48bd. Latest consensus state: current_term: 1 leader_uuid: "c7964e5424a54cf3a50405206fbc48bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } } }
07:27:33.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.550802 5778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [sys.catalog]: This master's current role is: FOLLOWER
07:27:33.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.551047 5789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7964e5424a54cf3a50405206fbc48bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } } }
07:27:33.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.552016 5789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212 [sys.catalog]: This master's current role is: FOLLOWER
07:27:33.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.559394 5736 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0: acquiring CA information for follower catalog manager: success
07:27:33.572 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.572108 5736 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 25ee7bac5dfe4bc58ca124c5def320b0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:27:33.628 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.628363 5806 catalog_manager.cc:1259] Loaded cluster ID: 62f0eb66c5714fbcbb050c988fc55bf5
07:27:33.628 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.628652 5806 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212: loading cluster ID for follower catalog manager: success
07:27:33.653 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.653446 5806 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212: acquiring CA information for follower catalog manager: success
07:27:33.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.665544 5806 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 19dd34855bc94452b795a818eb8d9212: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:27:33.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:33.771126 5774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:27:33.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:33.771834 5774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:27:33.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:33.772661 5774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:27:33.835 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:33.834831 5774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:27:33.836 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:33.836222 5774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.52.65
07:27:33.902 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.900210 5759 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7964e5424a54cf3a50405206fbc48bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19dd34855bc94452b795a818eb8d9212" member_type: VOTER last_known_addr { host: "127.5.52.126" port: 33565 } } peers { permanent_uuid: "c7964e5424a54cf3a50405206fbc48bd" member_type: VOTER last_known_addr { host: "127.5.52.125" port: 45789 } } peers { permanent_uuid: "25ee7bac5dfe4bc58ca124c5def320b0" member_type: VOTER last_known_addr { host: "127.5.52.124" port: 38815 } } }
07:27:33.902 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.902408 5774 tablet_server_runner.cc:78] Tablet server non-default flags:
07:27:33.902 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:33.902 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.5.52.84:41079
07:27:33.902 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/data
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/wal
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.5.52.65:0
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/data/info.pb
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.5.52.65
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.5.52.126:33565,127.5.52.125:45789,127.5.52.124:38815
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.5.52.65
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/logs
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 07 Aug 2026 06:41:47 UTC on ccycloud.quasar-byjrmo.root.comops.site
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4571901
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:27:33.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.902292 5759 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7964e5424a54cf3a50405206fbc48bd [sys.catalog]: This master's current role is: LEADER
07:27:33.904 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.904105 5774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:27:33.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:33.906147 5774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:27:33.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:33.956718 5829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:35.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:33.969304 5832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:35.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:35.587358 5830 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
07:27:35.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.587605 5774 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
07:27:35.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:33.967360 5828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:35.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.594853 5774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:27:35.606 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.606164 5774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:27:35.608 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.607797 5774 hybrid_clock.cc:648] HybridClock initialized: now 1786087655607729 us; error 86 us; skew 500 ppm
07:27:35.609 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.609148 5774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:27:35.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.632776 5774 webserver.cc:460] Webserver started at http://127.5.52.65:44319/ using document root <none> and password file <none>
07:27:35.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.634627 5774 fs_manager.cc:359] Metadata directory not provided
07:27:35.635 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.635490 5774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:27:35.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.636193 5774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:27:35.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.644172 5774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/data/instance:
07:27:35.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8b7db66be0e948cd87ca279cc80824bc"
07:27:35.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:35 on dist-test-slave-7nm7"
07:27:35.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.646693 5774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/wal/instance:
07:27:35.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8b7db66be0e948cd87ca279cc80824bc"
07:27:35.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:35 on dist-test-slave-7nm7"
07:27:35.663 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.662875 5774 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.013s sys 0.000s
07:27:35.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.677743 5840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:27:35.680 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.679849 5774 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.005s sys 0.004s
07:27:35.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.680671 5774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/data,/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/wal
07:27:35.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8b7db66be0e948cd87ca279cc80824bc"
07:27:35.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:35 on dist-test-slave-7nm7"
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.681424 5774 fs_report.cc:383] FS layout report
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/wal
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/wal
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/data/data
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:27:35.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:27:35.770 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.769773 5774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:27:35.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.772145 5774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:27:35.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.772912 5774 kserver.cc:163] Server-wide thread pool size limit: 3276
07:27:35.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.782610 5774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
07:27:35.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.795286 5774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
07:27:35.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.795809 5774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
07:27:35.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.796247 5774 ts_tablet_manager.cc:610] Registered 0 tablets
07:27:35.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:35.796557 5774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
07:27:36.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.120565 5774 rpc_server.cc:307] RPC server started. Bound to: 127.5.52.65:45563
07:27:36.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.120674 6028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.52.65:45563 every 8 connection(s)
07:27:36.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.123471 5774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/data/info.pb
07:27:36.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.128468 5329 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu as pid 5774
07:27:36.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.129040 5329 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-0/wal/instance
07:27:36.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.153916 5329 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:36.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:36.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/wal
07:27:36.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/data
07:27:36.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:27:36.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:27:36.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:27:36.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:36.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:36.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:36.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/logs
07:27:36.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/data/info.pb
07:27:36.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:36.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:27:36.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:27:36.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:27:36.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:27:36.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:36.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:36.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:36.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
07:27:36.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:27:36.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.5.52.66:0
07:27:36.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.5.52.66
07:27:36.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.5.52.66
07:27:36.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:36.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.5.52.126:33565,127.5.52.125:45789,127.5.52.124:38815
07:27:36.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.5.52.84:41079
07:27:36.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:36.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:36.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
07:27:36.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.219985 6034 heartbeater.cc:347] Connected to a master server at 127.5.52.125:45789
07:27:36.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.220484 6034 heartbeater.cc:464] Registering TS with master...
07:27:36.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.221812 6034 heartbeater.cc:511] Master 127.5.52.125:45789 requested a full tablet report, sending...
07:27:36.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.223289 6033 heartbeater.cc:347] Connected to a master server at 127.5.52.124:38815
07:27:36.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.223867 6033 heartbeater.cc:464] Registering TS with master...
07:27:36.229 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.225116 5516 ts_manager.cc:194] Registered new tserver with Master: 8b7db66be0e948cd87ca279cc80824bc (127.5.52.65:45563)
07:27:36.229 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.225239 6033 heartbeater.cc:511] Master 127.5.52.124:38815 requested a full tablet report, sending...
07:27:36.229 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.227947 5516 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.52.65:50721
07:27:36.229 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.228369 5643 ts_manager.cc:194] Registered new tserver with Master: 8b7db66be0e948cd87ca279cc80824bc (127.5.52.65:45563)
07:27:36.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.235790 6036 heartbeater.cc:347] Connected to a master server at 127.5.52.126:33565
07:27:36.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.236128 6036 heartbeater.cc:464] Registering TS with master...
07:27:36.237 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.236829 6036 heartbeater.cc:511] Master 127.5.52.126:33565 requested a full tablet report, sending...
07:27:36.239 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.239501 5398 ts_manager.cc:194] Registered new tserver with Master: 8b7db66be0e948cd87ca279cc80824bc (127.5.52.65:45563)
07:27:36.570 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:36.570468 6043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:27:36.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:36.570950 6043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:27:36.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:36.571410 6043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:27:36.606 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:36.605755 6043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:27:36.606 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:36.606712 6043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.52.66
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.646667 6043 tablet_server_runner.cc:78] Tablet server non-default flags:
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.5.52.84:41079
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/data
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/wal
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.5.52.66:0
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/data/info.pb
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.5.52.66
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.5.52.126:33565,127.5.52.125:45789,127.5.52.124:38815
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:36.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:27:36.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.5.52.66
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/logs
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 07 Aug 2026 06:41:47 UTC on ccycloud.quasar-byjrmo.root.comops.site
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4571901
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:27:36.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.648510 6043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:27:36.650 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:36.650175 6043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:27:36.673 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:36.673123 6061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:37.233 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.233310 6034 heartbeater.cc:503] Master 127.5.52.125:45789 was elected leader, sending a full tablet report...
07:27:37.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:36.673172 6056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:37.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:36.673293 6057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:37.940 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:37.940037 6058 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
07:27:37.940 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.940152 6043 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
07:27:37.945 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.945429 6043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:27:37.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.948500 6043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:27:37.950 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.949946 6043 hybrid_clock.cc:648] HybridClock initialized: now 1786087657949917 us; error 49 us; skew 500 ppm
07:27:37.950 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.950791 6043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:27:37.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.958541 6043 webserver.cc:460] Webserver started at http://127.5.52.66:42053/ using document root <none> and password file <none>
07:27:37.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.959416 6043 fs_manager.cc:359] Metadata directory not provided
07:27:37.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.959605 6043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:27:37.960 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.960000 6043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:27:37.965 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.965109 6043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/data/instance:
07:27:37.965 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "6392c02c5c3c469683079193c8bad0dc"
07:27:37.965 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:37 on dist-test-slave-7nm7"
07:27:37.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.966144 6043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/wal/instance:
07:27:37.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "6392c02c5c3c469683079193c8bad0dc"
07:27:37.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:37 on dist-test-slave-7nm7"
07:27:37.973 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.973250 6043 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
07:27:37.979 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.979113 6076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:27:37.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.980113 6043 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
07:27:37.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.980458 6043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/data,/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/wal
07:27:37.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "6392c02c5c3c469683079193c8bad0dc"
07:27:37.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:37 on dist-test-slave-7nm7"
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:37.980770 6043 fs_report.cc:383] FS layout report
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/wal
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/wal
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/data/data
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:27:37.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:27:38.043 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.043035 6043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:27:38.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.045022 6043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:27:38.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.045499 6043 kserver.cc:163] Server-wide thread pool size limit: 3276
07:27:38.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.048559 6043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
07:27:38.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.052889 6043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
07:27:38.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.053102 6043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
07:27:38.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.053304 6043 ts_tablet_manager.cc:610] Registered 0 tablets
07:27:38.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.053433 6043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
07:27:38.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.224397 6043 rpc_server.cc:307] RPC server started. Bound to: 127.5.52.66:39891
07:27:38.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.224510 6242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.52.66:39891 every 8 connection(s)
07:27:38.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.227095 6043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/data/info.pb
07:27:38.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.233858 5329 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu as pid 6043
07:27:38.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.235018 5329 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-1/wal/instance
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.259363 5329 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/wal
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/data
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/logs
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/data/info.pb
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.5.52.67:0
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.5.52.67
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.5.52.67
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.5.52.126:33565,127.5.52.125:45789,127.5.52.124:38815
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.5.52.84:41079
07:27:38.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:38.261 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:38.261 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
07:27:38.283 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.282785 6245 heartbeater.cc:347] Connected to a master server at 127.5.52.126:33565
07:27:38.283 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.283362 6245 heartbeater.cc:464] Registering TS with master...
07:27:38.285 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.284843 6244 heartbeater.cc:347] Connected to a master server at 127.5.52.125:45789
07:27:38.285 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.284942 6245 heartbeater.cc:511] Master 127.5.52.126:33565 requested a full tablet report, sending...
07:27:38.285 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.285349 6244 heartbeater.cc:464] Registering TS with master...
07:27:38.286 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.286135 6244 heartbeater.cc:511] Master 127.5.52.125:45789 requested a full tablet report, sending...
07:27:38.289 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.287104 6243 heartbeater.cc:347] Connected to a master server at 127.5.52.124:38815
07:27:38.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.287472 6243 heartbeater.cc:464] Registering TS with master...
07:27:38.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.288269 6243 heartbeater.cc:511] Master 127.5.52.124:38815 requested a full tablet report, sending...
07:27:38.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.288507 5516 ts_manager.cc:194] Registered new tserver with Master: 6392c02c5c3c469683079193c8bad0dc (127.5.52.66:39891)
07:27:38.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.288899 5398 ts_manager.cc:194] Registered new tserver with Master: 6392c02c5c3c469683079193c8bad0dc (127.5.52.66:39891)
07:27:38.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.290506 5516 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.52.66:59869
07:27:38.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.291046 5643 ts_manager.cc:194] Registered new tserver with Master: 6392c02c5c3c469683079193c8bad0dc (127.5.52.66:39891)
07:27:38.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:38.339963 5687 debug-util.cc:398] Leaking SignalData structure 0x7b08000ba420 after lost signal to thread 5587
07:27:38.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:38.653834 6261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:27:38.654 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:38.654444 6261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:27:38.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:38.654994 6261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:27:38.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:38.689910 6261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:27:38.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:38.690871 6261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.52.67
07:27:38.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.727442 6261 tablet_server_runner.cc:78] Tablet server non-default flags:
07:27:38.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:38.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.5.52.84:41079
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/data
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/wal
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.5.52.67:0
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/data/info.pb
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.5.52.67
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.5.52.126:33565,127.5.52.125:45789,127.5.52.124:38815
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.5.52.67
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/logs
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 07 Aug 2026 06:41:47 UTC on ccycloud.quasar-byjrmo.root.comops.site
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4571901
07:27:38.728 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:27:38.729 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.728955 6261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:27:38.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:38.730712 6261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:27:38.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:38.767618 6269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:39.296 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:39.295504 6244 heartbeater.cc:503] Master 127.5.52.125:45789 was elected leader, sending a full tablet report...
07:27:39.406 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:39.406248 6235 debug-util.cc:398] Leaking SignalData structure 0x7b08000af060 after lost signal to thread 6048
07:27:39.407 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:39.407399 6235 debug-util.cc:398] Leaking SignalData structure 0x7b08000aff60 after lost signal to thread 6242
07:27:40.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:38.771237 6270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:40.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:38.767694 6272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:40.318 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:40.318150 6271 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1547 milliseconds
07:27:40.318 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.318295 6261 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
07:27:40.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.319633 6261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:27:40.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.322542 6261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:27:40.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.323973 6261 hybrid_clock.cc:648] HybridClock initialized: now 1786087660323913 us; error 78 us; skew 500 ppm
07:27:40.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.324852 6261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:27:40.333 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.333457 6261 webserver.cc:460] Webserver started at http://127.5.52.67:40077/ using document root <none> and password file <none>
07:27:40.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.338210 6261 fs_manager.cc:359] Metadata directory not provided
07:27:40.338 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.338548 6261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:27:40.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.339162 6261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:27:40.348 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.348174 6261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/data/instance:
07:27:40.348 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "fddbed46b095462caf140b8c59db2b79"
07:27:40.348 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:40 on dist-test-slave-7nm7"
07:27:40.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.349777 6261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/wal/instance:
07:27:40.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "fddbed46b095462caf140b8c59db2b79"
07:27:40.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:40 on dist-test-slave-7nm7"
07:27:40.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.359799 6261 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.001s
07:27:40.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.370774 6285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:27:40.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.372192 6261 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.004s
07:27:40.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.372613 6261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/data,/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/wal
07:27:40.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "fddbed46b095462caf140b8c59db2b79"
07:27:40.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:40 on dist-test-slave-7nm7"
07:27:40.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.373093 6261 fs_report.cc:383] FS layout report
07:27:40.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:27:40.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/wal
07:27:40.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/wal
07:27:40.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/data/data
07:27:40.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:27:40.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:27:40.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:27:40.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:27:40.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:27:40.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:27:40.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:27:40.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:27:40.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:27:40.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:27:40.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:27:40.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.461544 6261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:27:40.464 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.463728 6261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:27:40.464 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.464350 6261 kserver.cc:163] Server-wide thread pool size limit: 3276
07:27:40.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.468290 6261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
07:27:40.475 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.474818 6261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
07:27:40.475 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.475132 6261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
07:27:40.475 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.475493 6261 ts_tablet_manager.cc:610] Registered 0 tablets
07:27:40.475 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.475744 6261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
07:27:40.751 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.751132 6466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.52.67:44811 every 8 connection(s)
07:27:40.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.751904 6261 rpc_server.cc:307] RPC server started. Bound to: 127.5.52.67:44811
07:27:40.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.755999 6261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/data/info.pb
07:27:40.761 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.757997 5329 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu as pid 6261
07:27:40.762 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.758556 5329 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster9172338081406115007/ts-2/wal/instance
07:27:40.841 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.841012 6472 heartbeater.cc:347] Connected to a master server at 127.5.52.125:45789
07:27:40.841 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.841593 6472 heartbeater.cc:464] Registering TS with master...
07:27:40.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.843016 6472 heartbeater.cc:511] Master 127.5.52.125:45789 requested a full tablet report, sending...
07:27:40.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.849376 5514 ts_manager.cc:194] Registered new tserver with Master: fddbed46b095462caf140b8c59db2b79 (127.5.52.67:44811)
07:27:40.854 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.852651 5514 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.52.67:38937
07:27:40.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.873610 6478 heartbeater.cc:347] Connected to a master server at 127.5.52.126:33565
07:27:40.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.874035 6478 heartbeater.cc:464] Registering TS with master...
07:27:40.875 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.874863 6478 heartbeater.cc:511] Master 127.5.52.126:33565 requested a full tablet report, sending...
07:27:40.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.878152 5398 ts_manager.cc:194] Registered new tserver with Master: fddbed46b095462caf140b8c59db2b79 (127.5.52.67:44811)
07:27:40.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.892655 6467 heartbeater.cc:347] Connected to a master server at 127.5.52.124:38815
07:27:40.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.893080 6467 heartbeater.cc:464] Registering TS with master...
07:27:40.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.893857 6467 heartbeater.cc:511] Master 127.5.52.124:38815 requested a full tablet report, sending...
07:27:40.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.898111 5643 ts_manager.cc:194] Registered new tserver with Master: fddbed46b095462caf140b8c59db2b79 (127.5.52.67:44811)
07:27:40.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:40.906651 5329 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
07:27:40.922 [DEBUG - main] (MiniKuduCluster.java:195) Response:
07:27:40.923 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_masters {
}
07:27:40.924 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_masters {
masters {
id {
type: MASTER
index: 0
}
bound_rpc_address {
host: "127.5.52.126"
port: 33565
}
bound_http_address {
host: "127.5.52.126"
port: 43957
}
}
masters {
id {
type: MASTER
index: 1
}
bound_rpc_address {
host: "127.5.52.125"
port: 45789
}
bound_http_address {
host: "127.5.52.125"
port: 34933
}
}
masters {
id {
type: MASTER
index: 2
}
bound_rpc_address {
host: "127.5.52.124"
port: 38815
}
bound_http_address {
host: "127.5.52.124"
port: 41541
}
}
}
07:27:40.925 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_tservers {
}
07:27:40.926 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_tservers {
tservers {
id {
type: TSERVER
index: 0
}
bound_rpc_address {
host: "127.5.52.65"
port: 45563
}
bound_http_address {
host: "127.5.52.65"
port: 44319
}
}
tservers {
id {
type: TSERVER
index: 1
}
bound_rpc_address {
host: "127.5.52.66"
port: 39891
}
bound_http_address {
host: "127.5.52.66"
port: 42053
}
}
tservers {
id {
type: TSERVER
index: 2
}
bound_rpc_address {
host: "127.5.52.67"
port: 44811
}
bound_http_address {
host: "127.5.52.67"
port: 40077
}
}
}
07:27:40.926 [INFO - main] (KuduTestHarness.java:157) Creating a new Kudu client...
07:27:40.927 [DEBUG - main] (SecurityUtil.java:105) Using ticketCache: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster5722852263861183523/krb5kdc/krb5cc
07:27:40.928 [DEBUG - main] (SecurityUtil.java:122) Could not login via JAAS. Using no credentials: Unable to obtain Principal Name for authentication
07:27:40.929 [INFO - Time-limited test] (TempDirUtils.java:71) Using the temp directory defined by TEST_TMPDIR: /tmp/dist-test-taskO_PLPU/test-tmp
07:27:40.930 [INFO - Time-limited test] (KuduBinaryLocator.java:83) Using Kudu binary directory specified by system property 'kuduBinDir': ../../../build/tsan/bin
07:27:40.930 [INFO - Time-limited test] (MiniKuduCluster.java:217) Starting process: [/tmp/dist-test-taskO_PLPU/java/build/dist-test/../../../build/tsan/bin/kudu, test, mini_cluster, --serialization=pb]
07:27:40.932 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: create_cluster {
num_masters: 3
num_tservers: 0
enable_kerberos: true
cluster_root: "/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609"
extra_master_flags: "--authn_token_validity_seconds=10"
hms_mode: NONE
mini_kdc_options {
ticket_lifetime: "10s"
renew_lifetime: "20s"
}
principal: "kudu"
mini_oidc_options {
}
}
07:27:41.447 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:27:41.447 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_cluster {
}
07:27:41.450 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:41.450521 6487 mini_oidc.cc:152] Starting JWKS server
07:27:41.455 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:41.454829 6487 webserver.cc:460] Webserver started at http://127.0.0.1:40849/ using document root <none> and password file <none>
07:27:41.455 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:41.455322 6487 mini_oidc.cc:189] Starting OIDC Discovery server
07:27:41.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:41.457685 6487 webserver.cc:460] Webserver started at http://127.0.0.1:35951/ using document root <none> and password file <none>
07:27:41.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Loading random data
07:27:41.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Initializing database '/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/principal' for realm 'KRBTEST.COM',
07:27:41.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master key name 'K/M@KRBTEST.COM'
07:27:41.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:41 dist-test-slave-7nm7 krb5kdc[6512](info): setting up network...
07:27:41.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
07:27:41.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:41 dist-test-slave-7nm7 krb5kdc[6512](info): set up 2 sockets
07:27:41.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:41 dist-test-slave-7nm7 krb5kdc[6512](info): commencing operation
07:27:41.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) krb5kdc: starting...
07:27:41.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:41.858618 6472 heartbeater.cc:503] Master 127.5.52.125:45789 was elected leader, sending a full tablet report...
07:27:42.851 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:42.850951 6487 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.011s sys 0.022s
07:27:42.869 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
07:27:42.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
07:27:42.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "test-admin@KRBTEST.COM" created.
07:27:42.901 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
07:27:42.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
07:27:42.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "test-user@KRBTEST.COM" created.
07:27:42.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
07:27:42.945 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
07:27:42.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "joe-interloper@KRBTEST.COM" created.
07:27:42.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
07:27:42.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/test-user.keytab.
07:27:42.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/test-user.keytab.
07:27:42.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:42 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:27:42.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Password for test-admin@KRBTEST.COM:
07:27:43.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-07T07:27:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
07:27:43.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-07T07:27:43Z Disabled control of system clock
07:27:43.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for kudu/127.6.85.254@KRBTEST.COM; defaulting to no policy
07:27:43.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
07:27:43.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "kudu/127.6.85.254@KRBTEST.COM" created.
07:27:43.073 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
07:27:43.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.6.85.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.254.keytab.
07:27:43.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.6.85.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.254.keytab.
07:27:43.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:43.076421 6487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:43.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:43.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/wal
07:27:43.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data
07:27:43.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:27:43.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:27:43.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:27:43.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:43.076 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/logs
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data/info.pb
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.6.85.254:43853
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.6.85.254
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.254.keytab
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.6.85.254
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.6.85.212:38715
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:43.077 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.6.85.254:43853,127.6.85.253:35731,127.6.85.252:45547
07:27:43.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:43.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:40849/jwks.json
07:27:43.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10 with env {KRB5CCNAME=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
07:27:43.409 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.408880 6531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:27:43.409 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.409605 6531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:27:43.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.410727 6531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:27:43.446 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.445984 6531 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=10
07:27:43.446 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.446318 6531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:27:43.446 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.446749 6531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:27:43.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.447103 6531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:27:43.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.447398 6531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:27:43.447 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.447691 6531 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:27:43.448 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.447909 6531 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:40849/jwks.json
07:27:43.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:43.487375 6531 master_runner.cc:386] Master server non-default flags:
07:27:43.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:43.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.6.85.212:38715
07:27:43.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:43.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/wal
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.6.85.254:43853,127.6.85.253:35731,127.6.85.252:45547
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.254.keytab
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.6.85.254
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.6.85.254:43853
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:40849/jwks.json
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=<redacted>
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data/info.pb
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.6.85.254
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/logs
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 07 Aug 2026 06:41:47 UTC on ccycloud.quasar-byjrmo.root.comops.site
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4571901
07:27:43.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:27:43.489 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:43.488894 6531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:27:43.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:43.490042 6531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:27:43.511 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.510623 6542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:44.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.512216 6540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:44.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:43.513504 6531 server_base.cc:1034] running on GCE node
07:27:44.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:43.514357 6545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:44.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.806006 6531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:27:44.810 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.809710 6531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:27:44.811 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.811306 6531 hybrid_clock.cc:648] HybridClock initialized: now 1786087664811180 us; error 143 us; skew 500 ppm
07:27:44.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.812325 6531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:27:44.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:44 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087664, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:27:44.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.818778 6531 init.cc:377] Logged in from keytab as kudu/127.6.85.254@KRBTEST.COM (short username kudu)
07:27:44.825 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.825402 6531 webserver.cc:460] Webserver started at http://127.6.85.254:38499/ using document root <none> and password file <none>
07:27:44.826 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.826454 6531 fs_manager.cc:359] Metadata directory not provided
07:27:44.826 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.826705 6531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:27:44.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.827188 6531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:27:44.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.832769 6531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data/instance:
07:27:44.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "90060c701cec4f34999ca0708ffc3e6d"
07:27:44.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:44 on dist-test-slave-7nm7"
07:27:44.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.834617 6531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/wal/instance:
07:27:44.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "90060c701cec4f34999ca0708ffc3e6d"
07:27:44.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:44 on dist-test-slave-7nm7"
07:27:44.842 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.841984 6531 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
07:27:44.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.848284 6564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:27:44.849 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.849397 6531 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.849694 6531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data,/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/wal
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "90060c701cec4f34999ca0708ffc3e6d"
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:44 on dist-test-slave-7nm7"
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.849993 6531 fs_report.cc:383] FS layout report
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/wal
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/wal
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data/data
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:27:44.850 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:27:44.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.884519 6531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:27:44.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.890516 6531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:27:44.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.890946 6531 kserver.cc:163] Server-wide thread pool size limit: 3276
07:27:44.970 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.970330 6531 rpc_server.cc:307] RPC server started. Bound to: 127.6.85.254:43853
07:27:44.971 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.970794 6651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.85.254:43853 every 8 connection(s)
07:27:44.973 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.973266 6531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data/info.pb
07:27:44.979 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.979547 6653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
07:27:44.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.981371 6487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu as pid 6531
07:27:44.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.981870 6487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/wal/instance
07:27:44.994 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:44.991887 6653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } has no permanent_uuid. Determining permanent_uuid...
07:27:45.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for kudu/127.6.85.253@KRBTEST.COM; defaulting to no policy
07:27:45.007 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
07:27:45.007 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "kudu/127.6.85.253@KRBTEST.COM" created.
07:27:45.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:45 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087664, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.254@KRBTEST.COM for kudu/127.6.85.254@KRBTEST.COM
07:27:45.040 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
07:27:45.040 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.6.85.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.253.keytab.
07:27:45.040 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.6.85.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.253.keytab.
07:27:45.042 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:45.041455 6653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } has no permanent_uuid. Determining permanent_uuid...
07:27:45.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:45.043929 6487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:45.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:45.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/wal
07:27:45.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data
07:27:45.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:27:45.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:27:45.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:27:45.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:45.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:45.044 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/logs
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data/info.pb
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.6.85.253:35731
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.6.85.253
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.253.keytab
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.6.85.253
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.6.85.212:38715
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.6.85.254:43853,127.6.85.253:35731,127.6.85.252:45547
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:40849/jwks.json
07:27:45.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10 with env {KRB5CCNAME=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
07:27:45.046 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.045821 6565 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111)
07:27:45.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.050534 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.253:35731: Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111)
07:27:45.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:45.100473 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } attempt: 1
07:27:45.106 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.106292 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.253:35731: Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111)
07:27:45.175 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:45.174785 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } attempt: 2
07:27:45.179 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.179405 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.253:35731: Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111)
07:27:45.271 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:45.270889 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } attempt: 3
07:27:45.275 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.274874 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.253:35731: Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111)
07:27:45.418 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.418202 6666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:27:45.418 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:45.418334 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } attempt: 4
07:27:45.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.419273 6666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:27:45.420 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.420405 6666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:27:45.424 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.424290 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.253:35731: Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111)
07:27:45.459 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.459039 6666 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=10
07:27:45.459 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.459466 6666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:27:45.460 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.459931 6666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:27:45.460 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.460265 6666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:27:45.460 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.460683 6666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:27:45.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.460994 6666 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:27:45.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.461252 6666 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:40849/jwks.json
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:45.502871 6666 master_runner.cc:386] Master server non-default flags:
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.6.85.212:38715
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/wal
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.6.85.254:43853,127.6.85.253:35731,127.6.85.252:45547
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.253.keytab
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.6.85.253
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.6.85.253:35731
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:40849/jwks.json
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=<redacted>
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:45.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data/info.pb
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.6.85.253
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/logs
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:27:45.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:27:45.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:27:45.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:27:45.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:27:45.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 07 Aug 2026 06:41:47 UTC on ccycloud.quasar-byjrmo.root.comops.site
07:27:45.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4571901
07:27:45.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:27:45.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:45.504673 6666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:27:45.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:45.506048 6666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:27:45.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.527853 6680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:45.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:45.723764 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } attempt: 5
07:27:45.729 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.729152 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.253:35731: Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111)
07:27:46.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.276619 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } attempt: 6
07:27:46.285 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:46.284894 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.253:35731: Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111)
07:27:46.860 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.530118 6677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:46.860 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:45.531003 6685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:46.860 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:45.531235 6666 server_base.cc:1034] running on GCE node
07:27:46.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.865456 6666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:27:46.869 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.868788 6666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:27:46.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.870340 6666 hybrid_clock.cc:648] HybridClock initialized: now 1786087666870278 us; error 50 us; skew 500 ppm
07:27:46.871 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.871183 6666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:27:46.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:46 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087666, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:27:46.882 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.882617 6666 init.cc:377] Logged in from keytab as kudu/127.6.85.253@KRBTEST.COM (short username kudu)
07:27:46.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.888528 6666 webserver.cc:460] Webserver started at http://127.6.85.253:33219/ using document root <none> and password file <none>
07:27:46.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.889611 6666 fs_manager.cc:359] Metadata directory not provided
07:27:46.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.889827 6666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:27:46.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.890647 6666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:27:46.897 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.896719 6666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data/instance:
07:27:46.897 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "b257a17f1bcd4e8d8bb061d0230d2529"
07:27:46.897 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:46 on dist-test-slave-7nm7"
07:27:46.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.897873 6666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/wal/instance:
07:27:46.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "b257a17f1bcd4e8d8bb061d0230d2529"
07:27:46.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:46 on dist-test-slave-7nm7"
07:27:46.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.906810 6666 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
07:27:46.915 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.914695 6709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:27:46.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.916260 6666 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
07:27:46.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.916656 6666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data,/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/wal
07:27:46.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "b257a17f1bcd4e8d8bb061d0230d2529"
07:27:46.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:46 on dist-test-slave-7nm7"
07:27:46.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.917037 6666 fs_report.cc:383] FS layout report
07:27:46.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:27:46.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/wal
07:27:46.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/wal
07:27:46.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data/data
07:27:46.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:27:46.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:27:46.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:27:46.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:27:46.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:27:46.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:27:46.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:27:46.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:27:46.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:27:46.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:27:46.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:27:46.976 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.976188 6666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:27:46.984 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.983832 6666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:27:46.984 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:46.984392 6666 kserver.cc:163] Server-wide thread pool size limit: 3276
07:27:47.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.073958 6666 rpc_server.cc:307] RPC server started. Bound to: 127.6.85.253:35731
07:27:47.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.074065 6806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.85.253:35731 every 8 connection(s)
07:27:47.078 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.077751 6666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data/info.pb
07:27:47.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.084982 6807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
07:27:47.086 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.085945 6487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu as pid 6666
07:27:47.086 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.086440 6487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/wal/instance
07:27:47.104 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.101485 6807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } has no permanent_uuid. Determining permanent_uuid...
07:27:47.108 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for kudu/127.6.85.252@KRBTEST.COM; defaulting to no policy
07:27:47.112 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
07:27:47.112 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "kudu/127.6.85.252@KRBTEST.COM" created.
07:27:47.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:47 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087666, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.253@KRBTEST.COM for kudu/127.6.85.254@KRBTEST.COM
07:27:47.148 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.147984 6807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } has no permanent_uuid. Determining permanent_uuid...
07:27:47.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
07:27:47.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.6.85.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.252.keytab.
07:27:47.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.6.85.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.252.keytab.
07:27:47.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.156351 6487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:47.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:47.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/wal
07:27:47.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data
07:27:47.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/logs
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data/info.pb
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.6.85.252:45547
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.6.85.252
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:47.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.252.keytab
07:27:47.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.6.85.252
07:27:47.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:27:47.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
07:27:47.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
07:27:47.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.6.85.212:38715
07:27:47.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:47.158 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:47.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:47.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:47.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.6.85.254:43853,127.6.85.253:35731,127.6.85.252:45547
07:27:47.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:47.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:40849/jwks.json
07:27:47.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10 with env {KRB5CCNAME=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
07:27:47.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:47 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087666, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.253@KRBTEST.COM for kudu/127.6.85.253@KRBTEST.COM
07:27:47.181 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.180917 6807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } has no permanent_uuid. Determining permanent_uuid...
07:27:47.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.183688 6713 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:47.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.189016 6807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:47.239 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.238653 6807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 1
07:27:47.243 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.242915 6807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:47.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.311393 6807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 2
07:27:47.315 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.315601 6807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:47.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.345420 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } attempt: 7
07:27:47.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:47 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087664, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.254@KRBTEST.COM for kudu/127.6.85.253@KRBTEST.COM
07:27:47.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.374341 6653 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } has no permanent_uuid. Determining permanent_uuid...
07:27:47.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.380792 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:47.409 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.407086 6807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 3
07:27:47.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.413868 6807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:47.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.439327 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 1
07:27:47.444 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.443635 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:47.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.525115 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 2
07:27:47.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.529526 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:47.535 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.534737 6820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:27:47.535 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.535390 6820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:27:47.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.536085 6820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:27:47.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.557717 6807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 4
07:27:47.562 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.561712 6807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:47.574 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.573930 6820 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=10
07:27:47.574 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.574185 6820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:27:47.574 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.574574 6820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:27:47.574 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.574790 6820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:27:47.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.575055 6820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:27:47.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.575388 6820 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:27:47.575 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.575564 6820 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:40849/jwks.json
07:27:47.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.615010 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 3
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.619738 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.619505 6820 master_runner.cc:386] Master server non-default flags:
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.6.85.212:38715
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/wal
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.6.85.254:43853,127.6.85.253:35731,127.6.85.252:45547
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.252.keytab
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.6.85.252
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.6.85.252:45547
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:40849/jwks.json
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=<redacted>
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data/info.pb
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.6.85.252
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/logs
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:47.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:27:47.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:27:47.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:27:47.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:27:47.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:27:47.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:27:47.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 07 Aug 2026 06:41:47 UTC on ccycloud.quasar-byjrmo.root.comops.site
07:27:47.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4571901
07:27:47.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:27:47.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.620890 6820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:27:47.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.621872 6820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:27:47.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.641288 6838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:47.760 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.760182 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 4
07:27:47.765 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.764672 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:47.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.861294 6807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 5
07:27:47.866 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.865716 6807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:48.050 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.048699 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 5
07:27:48.054 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:48.054158 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:48.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.413168 6807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 6
07:27:48.425 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:48.424795 6807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:48.607 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.606825 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 6
07:27:48.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:48.614168 6653 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.85.252:45547: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:48.940 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.641409 6833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:48.940 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:47.641556 6834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:48.940 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:47.644421 6820 server_base.cc:1034] running on GCE node
07:27:48.945 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.944759 6820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:27:48.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.947599 6820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:27:48.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.948988 6820 hybrid_clock.cc:648] HybridClock initialized: now 1786087668948966 us; error 57 us; skew 500 ppm
07:27:48.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.949826 6820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:27:48.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:48 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087668, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:27:48.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.956383 6820 init.cc:377] Logged in from keytab as kudu/127.6.85.252@KRBTEST.COM (short username kudu)
07:27:48.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.961407 6820 webserver.cc:460] Webserver started at http://127.6.85.252:41365/ using document root <none> and password file <none>
07:27:48.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.962428 6820 fs_manager.cc:359] Metadata directory not provided
07:27:48.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.962637 6820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:27:48.963 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.963064 6820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:27:48.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.968384 6820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data/instance:
07:27:48.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "50bb959efc9a441286c37e330a3a9dc3"
07:27:48.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:48 on dist-test-slave-7nm7"
07:27:48.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.969566 6820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/wal/instance:
07:27:48.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "50bb959efc9a441286c37e330a3a9dc3"
07:27:48.969 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:48 on dist-test-slave-7nm7"
07:27:48.978 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.978042 6820 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.010s sys 0.000s
07:27:48.987 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.986608 6861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:27:48.988 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.988329 6820 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
07:27:48.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.988693 6820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data,/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/wal
07:27:48.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "50bb959efc9a441286c37e330a3a9dc3"
07:27:48.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:48 on dist-test-slave-7nm7"
07:27:48.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:48.989118 6820 fs_report.cc:383] FS layout report
07:27:48.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:27:48.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/wal
07:27:48.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/wal
07:27:48.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data/data
07:27:48.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:27:48.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:27:48.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:27:48.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:27:48.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:27:48.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:27:48.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:27:48.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:27:48.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:27:48.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:27:48.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:27:49.035 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.034938 6820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:27:49.041 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.041519 6820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:27:49.042 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.042075 6820 kserver.cc:163] Server-wide thread pool size limit: 3276
07:27:49.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.128274 6820 rpc_server.cc:307] RPC server started. Bound to: 127.6.85.252:45547
07:27:49.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.128312 6934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.85.252:45547 every 8 connection(s)
07:27:49.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.132035 6820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data/info.pb
07:27:49.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.137751 6487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu as pid 6820
07:27:49.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.138294 6487 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/wal/instance
07:27:49.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.139243 6935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
07:27:49.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.154060 6935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } has no permanent_uuid. Determining permanent_uuid...
07:27:49.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:49 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.85.252@KRBTEST.COM
07:27:49.178 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:49 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087668, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.252@KRBTEST.COM for kudu/127.6.85.254@KRBTEST.COM
07:27:49.193 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.193226 6935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } has no permanent_uuid. Determining permanent_uuid...
07:27:49.212 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:49 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087668, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.252@KRBTEST.COM for kudu/127.6.85.253@KRBTEST.COM
07:27:49.216 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:49 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.85.253@KRBTEST.COM
07:27:49.225 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.225502 6935 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } has no permanent_uuid. Determining permanent_uuid...
07:27:49.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:49 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087668, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.252@KRBTEST.COM for kudu/127.6.85.252@KRBTEST.COM
07:27:49.241 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:49 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087662, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.85.254@KRBTEST.COM
07:27:49.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.257977 6935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: Bootstrap starting.
07:27:49.259 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.259572 6487 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
07:27:49.266 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.265642 6935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: Neither blocks nor log segments found. Creating new log.
07:27:49.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.268280 6935 log.cc:824] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: Log is configured to *not* fsync() on all Append() calls
07:27:49.271 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:27:49.271 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: get_kdc_env_vars {
}
07:27:49.273 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: get_kdc_env_vars {
env_vars {
key: "KRB5CCNAME"
value: "/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5cc"
}
env_vars {
key: "KRB5_CONFIG"
value: "/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5.conf"
}
env_vars {
key: "KRB5_KDC_PROFILE"
value: "/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kdc.conf"
}
env_vars {
key: "KUDU_ENABLE_KRB5_REALM_FIX"
value: "yes"
}
}
07:27:49.273 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: get_masters {
}
07:27:49.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.273803 6935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: No bootstrap required, opened a new log
07:27:49.274 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: get_masters {
masters {
id {
type: MASTER
index: 0
}
bound_rpc_address {
host: "127.6.85.254"
port: 43853
}
bound_http_address {
host: "127.6.85.254"
port: 38499
}
}
masters {
id {
type: MASTER
index: 1
}
bound_rpc_address {
host: "127.6.85.253"
port: 35731
}
bound_http_address {
host: "127.6.85.253"
port: 33219
}
}
masters {
id {
type: MASTER
index: 2
}
bound_rpc_address {
host: "127.6.85.252"
port: 45547
}
bound_http_address {
host: "127.6.85.252"
port: 41365
}
}
}
07:27:49.274 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: get_tservers {
}
07:27:49.275 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:27:49.275 [INFO - Time-limited test] (MiniKuduCluster.java:622) Running kinit for user test-admin
07:27:49.275 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: kinit {
username: "test-admin"
}
07:27:49.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.300809 6935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:49.301 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.301496 6935 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
07:27:49.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.301812 6935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50bb959efc9a441286c37e330a3a9dc3, State: Initialized, Role: FOLLOWER
07:27:49.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.302961 6935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:49.307 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.306370 6952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:49.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.307729 6952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: This master's current role is: FOLLOWER
07:27:49.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.308166 6935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: configured and running, proceeding with master startup.
07:27:49.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:49 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:27:49.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Password for test-admin@KRBTEST.COM:
07:27:49.321 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:27:49.322 [DEBUG - Time-limited test] (SecurityUtil.java:105) Using ticketCache: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5cc
07:27:49.323 [DEBUG - Time-limited test] (SecurityUtil.java:119) Logged in as subject: test-admin@KRBTEST.COM
07:27:49.323 [INFO - Time-limited test] (MiniKuduCluster.java:622) Running kinit for user test-user
07:27:49.323 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: kinit {
username: "test-user"
}
07:27:49.343 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:49.342959 6968 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
07:27:49.344 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:49.343735 6968 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
07:27:49.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.337875 6820 master_runner.cc:186] Error getting master registration for 127.6.85.254:43853: OK, instance_id { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" instance_seqno: 1786087664948695 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
07:27:49.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.365533 6820 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
07:27:49.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:49 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087669, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:27:49.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Password for test-user@KRBTEST.COM:
07:27:49.382 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:27:49.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.485221 6807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 7
07:27:49.499 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:49 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087666, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.253@KRBTEST.COM for kudu/127.6.85.252@KRBTEST.COM
07:27:49.521 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.521118 6807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: Bootstrap starting.
07:27:49.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.528399 6807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: Neither blocks nor log segments found. Creating new log.
07:27:49.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.530817 6807 log.cc:824] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: Log is configured to *not* fsync() on all Append() calls
07:27:49.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.536320 6807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: No bootstrap required, opened a new log
07:27:49.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:49.548950 6873 tablet.cc:2369] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
07:27:49.566 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.565207 6807 raft_consensus.cc:348] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:49.567 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.566325 6807 raft_consensus.cc:374] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
07:27:49.567 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.566615 6807 raft_consensus.cc:729] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b257a17f1bcd4e8d8bb061d0230d2529, State: Initialized, Role: FOLLOWER
07:27:49.568 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.567610 6807 consensus_queue.cc:260] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:49.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.570554 6979 sys_catalog.cc:455] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:49.571 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.571521 6979 sys_catalog.cc:458] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: This master's current role is: FOLLOWER
07:27:49.572 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.572042 6807 sys_catalog.cc:564] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: configured and running, proceeding with master startup.
07:27:49.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:49.604506 6997 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
07:27:49.605 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:49.605254 6997 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
07:27:49.635 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.597517 6666 master_runner.cc:186] Error getting master registration for 127.6.85.254:43853: OK, instance_id { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" instance_seqno: 1786087664948695 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
07:27:49.635 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.635481 6666 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
07:27:49.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.647943 6653 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } attempt: 7
07:27:49.664 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:49 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087664, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.254@KRBTEST.COM for kudu/127.6.85.252@KRBTEST.COM
07:27:49.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.687759 6653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: Bootstrap starting.
07:27:49.695 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.695072 6653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: Neither blocks nor log segments found. Creating new log.
07:27:49.697 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.697599 6653 log.cc:824] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: Log is configured to *not* fsync() on all Append() calls
07:27:49.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.703495 6653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: No bootstrap required, opened a new log
07:27:49.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.735808 6653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:49.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.738981 6653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
07:27:49.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.739557 6653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90060c701cec4f34999ca0708ffc3e6d, State: Initialized, Role: FOLLOWER
07:27:49.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.741879 6653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:49.743 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:49.742872 6721 tablet.cc:2369] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: Can't schedule compaction. Clean time has not been advanced past its initial value.
07:27:49.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.753482 6653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: configured and running, proceeding with master startup.
07:27:49.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.751592 7003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:49.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.754300 7003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: This master's current role is: FOLLOWER
07:27:49.768 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.768415 6952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
07:27:49.769 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.768992 6952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:49.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.772910 6612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50bb959efc9a441286c37e330a3a9dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90060c701cec4f34999ca0708ffc3e6d" is_pre_election: true
07:27:49.774 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.773802 6612 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50bb959efc9a441286c37e330a3a9dc3 in term 0.
07:27:49.778 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.775527 6867 leader_election.cc:302] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50bb959efc9a441286c37e330a3a9dc3, 90060c701cec4f34999ca0708ffc3e6d; no voters:
07:27:49.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.779978 6756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50bb959efc9a441286c37e330a3a9dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" is_pre_election: true
07:27:49.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.780861 6756 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50bb959efc9a441286c37e330a3a9dc3 in term 0.
07:27:49.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.782362 6952 leader_election.cc:288] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90060c701cec4f34999ca0708ffc3e6d (127.6.85.254:43853), b257a17f1bcd4e8d8bb061d0230d2529 (127.6.85.253:35731)
07:27:49.783 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.783020 6952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
07:27:49.783 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.783514 6952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
07:27:49.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.784039 6952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 0 FOLLOWER]: Advancing to term 1
07:27:49.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.795748 6952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:49.810 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.804011 6612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50bb959efc9a441286c37e330a3a9dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90060c701cec4f34999ca0708ffc3e6d"
07:27:49.810 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.804615 6612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 0 FOLLOWER]: Advancing to term 1
07:27:49.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.812001 6612 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50bb959efc9a441286c37e330a3a9dc3 in term 1.
07:27:49.813 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.813311 6867 leader_election.cc:302] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50bb959efc9a441286c37e330a3a9dc3, 90060c701cec4f34999ca0708ffc3e6d; no voters:
07:27:49.814 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.814536 7012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 1 FOLLOWER]: Leader election won for term 1
07:27:49.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.817806 6756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50bb959efc9a441286c37e330a3a9dc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b257a17f1bcd4e8d8bb061d0230d2529"
07:27:49.822 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.818420 6756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [term 0 FOLLOWER]: Advancing to term 1
07:27:49.826 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.825783 6756 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50bb959efc9a441286c37e330a3a9dc3 in term 1.
07:27:49.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.827349 6952 leader_election.cc:288] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [CANDIDATE]: Term 1 election: Requested vote from peers 90060c701cec4f34999ca0708ffc3e6d (127.6.85.254:43853), b257a17f1bcd4e8d8bb061d0230d2529 (127.6.85.253:35731)
07:27:49.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.829327 7012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 1 LEADER]: Becoming Leader. State: Replica: 50bb959efc9a441286c37e330a3a9dc3, State: Running, Role: LEADER
07:27:49.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.830441 7012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:49.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.845158 6952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50bb959efc9a441286c37e330a3a9dc3. Latest consensus state: current_term: 1 leader_uuid: "50bb959efc9a441286c37e330a3a9dc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:49.846 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.846412 6952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: This master's current role is: LEADER
07:27:49.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.853305 7017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
07:27:49.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.861047 7017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
07:27:49.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.886113 6756 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [term 1 FOLLOWER]: Refusing update from remote peer 50bb959efc9a441286c37e330a3a9dc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
07:27:49.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.888316 6952 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
07:27:49.892 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.892127 7027 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
07:27:49.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.901183 6612 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 1 FOLLOWER]: Refusing update from remote peer 50bb959efc9a441286c37e330a3a9dc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
07:27:49.903 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:49.902518 6576 tablet.cc:2369] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
07:27:49.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.904592 6952 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
07:27:49.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.932919 6979 sys_catalog.cc:455] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50bb959efc9a441286c37e330a3a9dc3. Latest consensus state: current_term: 1 leader_uuid: "50bb959efc9a441286c37e330a3a9dc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:49.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.936918 7003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50bb959efc9a441286c37e330a3a9dc3. Latest consensus state: current_term: 1 leader_uuid: "50bb959efc9a441286c37e330a3a9dc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:49.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.937721 7003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: This master's current role is: FOLLOWER
07:27:49.940 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.939994 6979 sys_catalog.cc:458] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: This master's current role is: FOLLOWER
07:27:49.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.943519 7024 mvcc.cc:205] Tried to move back new op lower bound from 7315815095831752704 to 7315815095637864448. Current Snapshot: MvccSnapshot[applied={T|T < 7315815095831752704}]
07:27:49.949 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.948993 7029 mvcc.cc:205] Tried to move back new op lower bound from 7315815095831752704 to 7315815095637864448. Current Snapshot: MvccSnapshot[applied={T|T < 7315815095831752704}]
07:27:49.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.957337 7031 mvcc.cc:205] Tried to move back new op lower bound from 7315815095831752704 to 7315815095637864448. Current Snapshot: MvccSnapshot[applied={T|T < 7315815095831752704}]
07:27:49.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.960160 7003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50bb959efc9a441286c37e330a3a9dc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:49.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.961086 7003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: This master's current role is: FOLLOWER
07:27:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.964262 6952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50bb959efc9a441286c37e330a3a9dc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.965063 6952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: This master's current role is: LEADER
07:27:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.964417 7012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50bb959efc9a441286c37e330a3a9dc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:49.966 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.965616 7012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: This master's current role is: LEADER
07:27:49.971 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.970117 6979 sys_catalog.cc:455] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50bb959efc9a441286c37e330a3a9dc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:49.971 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.971194 6979 sys_catalog.cc:458] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: This master's current role is: FOLLOWER
07:27:49.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.980085 7017 catalog_manager.cc:1347] Generated new cluster ID: 225aa75c70c5475baade6fcbc26a0340
07:27:49.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:49.980381 7017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
07:27:50.034 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:50.033674 7017 catalog_manager.cc:1370] Generated new certificate authority record
07:27:50.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:50.036224 7017 catalog_manager.cc:1504] Loading token signing keys...
07:27:50.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:50.065188 7017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: Generated new TSK 0
07:27:50.066 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:50.066718 7017 catalog_manager.cc:1514] Initializing in-progress tserver states...
07:27:50.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:50.611379 6997 catalog_manager.cc:1259] Loaded cluster ID: 225aa75c70c5475baade6fcbc26a0340
07:27:50.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:50.611682 6997 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: loading cluster ID for follower catalog manager: success
07:27:50.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:50.619607 6997 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: acquiring CA information for follower catalog manager: success
07:27:50.625 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:50.625355 6997 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:27:50.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:50.916146 7027 catalog_manager.cc:1259] Loaded cluster ID: 225aa75c70c5475baade6fcbc26a0340
07:27:50.916 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:50.916656 7027 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: loading cluster ID for follower catalog manager: success
07:27:50.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:50.928648 7027 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: acquiring CA information for follower catalog manager: success
07:27:50.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:50.934990 7027 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:27:52.384 [INFO - Time-limited test] (MiniKuduCluster.java:399) Killing master server 127.6.85.254:43853
07:27:52.385 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
id {
type: MASTER
index: 0
}
}
07:27:52.386 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:52.385975 6487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu with pid 6531
07:27:52.436 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:27:52.436 [INFO - Time-limited test] (MiniKuduCluster.java:399) Killing master server 127.6.85.252:45547
07:27:52.436 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
id {
type: MASTER
index: 2
}
}
07:27:52.437 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:52.436976 6487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu with pid 6820
07:27:52.484 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:27:52.484 [INFO - Time-limited test] (MiniKuduCluster.java:399) Killing master server 127.6.85.253:35731
07:27:52.484 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
id {
type: MASTER
index: 1
}
}
07:27:52.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:52.485241 6487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu with pid 6666
07:27:52.523 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:27:52.523 [INFO - Time-limited test] (MiniKuduCluster.java:380) Starting master server 127.6.85.254:43853
07:27:52.523 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
id {
type: MASTER
index: 0
}
}
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:52.525573 6487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/wal
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/logs
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data/info.pb
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.6.85.254:43853
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.6.85.254
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=38499
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.254.keytab
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.6.85.254
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
07:27:52.526 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
07:27:52.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.6.85.212:38715
07:27:52.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:52.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:52.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:52.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:52.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.6.85.254:43853,127.6.85.253:35731,127.6.85.252:45547
07:27:52.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:52.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:40849/jwks.json
07:27:52.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10 with env {KRB5CCNAME=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
07:27:52.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.863488 7040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:27:52.864 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.864370 7040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:27:52.865 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.865172 7040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:27:52.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.900259 7040 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=10
07:27:52.900 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.900554 7040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:27:52.901 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.901077 7040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:27:52.901 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.901458 7040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:27:52.902 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.901772 7040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:27:52.902 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.902060 7040 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:27:52.902 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.902313 7040 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:40849/jwks.json
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:52.942878 7040 master_runner.cc:386] Master server non-default flags:
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.6.85.212:38715
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/wal
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.6.85.254:43853,127.6.85.253:35731,127.6.85.252:45547
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.254.keytab
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.6.85.254
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.6.85.254:43853
07:27:52.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:40849/jwks.json
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=<redacted>
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data/info.pb
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.6.85.254
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=38499
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/logs
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 07 Aug 2026 06:41:47 UTC on ccycloud.quasar-byjrmo.root.comops.site
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4571901
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:27:52.944 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:52.944638 7040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:27:52.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:52.945880 7040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:27:52.967 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.967147 7053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:54.247 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.967182 7051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:54.247 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:52.968945 7040 server_base.cc:1034] running on GCE node
07:27:54.248 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:52.967558 7050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:54.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.251602 7040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:27:54.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.254664 7040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:27:54.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.256108 7040 hybrid_clock.cc:648] HybridClock initialized: now 1786087674256059 us; error 58 us; skew 500 ppm
07:27:54.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.256983 7040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:27:54.266 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:54 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087674, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:27:54.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.267504 7040 init.cc:377] Logged in from keytab as kudu/127.6.85.254@KRBTEST.COM (short username kudu)
07:27:54.272 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.272578 7040 webserver.cc:460] Webserver started at http://127.6.85.254:38499/ using document root <none> and password file <none>
07:27:54.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.273480 7040 fs_manager.cc:359] Metadata directory not provided
07:27:54.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.273761 7040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:27:54.282 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.282599 7040 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.004s
07:27:54.287 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.287657 7066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:27:54.289 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.288837 7040 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.005s
07:27:54.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.289211 7040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data,/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/wal
07:27:54.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "90060c701cec4f34999ca0708ffc3e6d"
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:44 on dist-test-slave-7nm7"
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.291893 7040 fs_report.cc:383] FS layout report
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/wal
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/wal
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data/data
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:27:54.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:27:54.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:27:54.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:27:54.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:27:54.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.338712 7040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:27:54.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.345317 7040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:27:54.346 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.345870 7040 kserver.cc:163] Server-wide thread pool size limit: 3276
07:27:54.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.452289 7040 rpc_server.cc:307] RPC server started. Bound to: 127.6.85.254:43853
07:27:54.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.452455 7145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.85.254:43853 every 8 connection(s)
07:27:54.455 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.455431 7040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-0/data/info.pb
07:27:54.462 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.462504 6487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu as pid 7040
07:27:54.463 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:27:54.463 [INFO - Time-limited test] (MiniKuduCluster.java:380) Starting master server 127.6.85.252:45547
07:27:54.463 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
id {
type: MASTER
index: 2
}
}
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.464813 6487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/wal
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/logs
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data/info.pb
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:54.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.6.85.252:45547
07:27:54.466 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.6.85.252
07:27:54.466 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=41365
07:27:54.466 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.252.keytab
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.6.85.252
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.6.85.212:38715
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.6.85.254:43853,127.6.85.253:35731,127.6.85.252:45547
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:40849/jwks.json
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10 with env {KRB5CCNAME=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
07:27:54.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.467005 7148 sys_catalog.cc:263] Verifying existing consensus state
07:27:54.474 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.474200 7148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: Bootstrap starting.
07:27:54.516 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.515908 7148 log.cc:824] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: Log is configured to *not* fsync() on all Append() calls
07:27:54.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.539240 7148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
07:27:54.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.540702 7148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: Bootstrap complete.
07:27:54.584 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.581714 7148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:54.584 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.584182 7148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90060c701cec4f34999ca0708ffc3e6d, State: Initialized, Role: FOLLOWER
07:27:54.585 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.585085 7148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:54.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.588325 7156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:54.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.589402 7156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: This master's current role is: FOLLOWER
07:27:54.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.590132 7148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: configured and running, proceeding with master startup.
07:27:54.640 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.640440 7173 catalog_manager.cc:1259] Loaded cluster ID: 225aa75c70c5475baade6fcbc26a0340
07:27:54.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.640748 7173 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: loading cluster ID for follower catalog manager: success
07:27:54.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:54 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087674, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.254@KRBTEST.COM for kudu/127.6.85.254@KRBTEST.COM
07:27:54.652 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.652012 7173 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: acquiring CA information for follower catalog manager: success
07:27:54.659 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.658718 7173 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:27:54.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.676563 7040 master_runner.cc:186] Error getting master registration for 127.6.85.253:35731: Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111),
07:27:54.709 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.709134 7040 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
07:27:54.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.847256 7152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:27:54.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.847810 7152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:27:54.848 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.848467 7152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:27:54.882 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.882522 7152 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=10
07:27:54.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.882866 7152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:27:54.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.883483 7152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:27:54.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.883715 7152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:27:54.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.883913 7152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:27:54.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.884101 7152 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:27:54.884 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.884250 7152 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:40849/jwks.json
07:27:54.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.933307 7152 master_runner.cc:386] Master server non-default flags:
07:27:54.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:54.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.6.85.212:38715
07:27:54.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:54.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:54.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/wal
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.6.85.254:43853,127.6.85.253:35731,127.6.85.252:45547
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.252.keytab
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.6.85.252
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.6.85.252:45547
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:40849/jwks.json
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=<redacted>
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data/info.pb
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.6.85.252
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=41365
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/logs
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:27:54.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:27:54.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:27:54.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 07 Aug 2026 06:41:47 UTC on ccycloud.quasar-byjrmo.root.comops.site
07:27:54.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4571901
07:27:54.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:27:54.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.934684 7152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:27:54.936 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.935799 7152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:27:54.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.957569 7188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:55.859 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:55.858733 7198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
07:27:55.860 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:55.859333 7198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:55.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:55.862821 7198 leader_election.cc:288] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b257a17f1bcd4e8d8bb061d0230d2529 (127.6.85.253:35731), 50bb959efc9a441286c37e330a3a9dc3 (127.6.85.252:45547)
07:27:55.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:55.868459 7071 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:55.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:55.880419 7071 leader_election.cc:334] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 50bb959efc9a441286c37e330a3a9dc3 (127.6.85.252:45547): Network error: Client connection negotiation failed: client connection to 127.6.85.252:45547: connect: Connection refused (error 111)
07:27:55.885 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:55.885147 7067 leader_election.cc:334] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b257a17f1bcd4e8d8bb061d0230d2529 (127.6.85.253:35731): Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111)
07:27:55.886 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:55.885637 7067 leader_election.cc:302] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 90060c701cec4f34999ca0708ffc3e6d; no voters: 50bb959efc9a441286c37e330a3a9dc3, b257a17f1bcd4e8d8bb061d0230d2529
07:27:55.887 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:55.886662 7198 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
07:27:56.327 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:54.961766 7152 server_base.cc:1034] running on GCE node
07:27:56.327 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.960914 7192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:56.328 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:54.961072 7187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:56.333 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.332780 7152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:27:56.336 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.336102 7152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:27:56.337 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.337608 7152 hybrid_clock.cc:648] HybridClock initialized: now 1786087676337531 us; error 83 us; skew 500 ppm
07:27:56.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.338810 7152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:27:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:56 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087676, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:27:56.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.346452 7152 init.cc:377] Logged in from keytab as kudu/127.6.85.252@KRBTEST.COM (short username kudu)
07:27:56.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.355923 7152 webserver.cc:460] Webserver started at http://127.6.85.252:41365/ using document root <none> and password file <none>
07:27:56.357 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.356926 7152 fs_manager.cc:359] Metadata directory not provided
07:27:56.357 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.357146 7152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:27:56.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.366515 7152 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.006s sys 0.001s
07:27:56.372 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.372126 7215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:27:56.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.374828 7152 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.375242 7152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data,/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/wal
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "50bb959efc9a441286c37e330a3a9dc3"
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:48 on dist-test-slave-7nm7"
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.378788 7152 fs_report.cc:383] FS layout report
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/wal
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/wal
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data/data
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:27:56.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:27:56.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:27:56.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:27:56.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:27:56.459 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.458840 7152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:27:56.466 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.466646 7152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:27:56.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.467208 7152 kserver.cc:163] Server-wide thread pool size limit: 3276
07:27:56.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.581079 7152 rpc_server.cc:307] RPC server started. Bound to: 127.6.85.252:45547
07:27:56.582 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.581837 7301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.85.252:45547 every 8 connection(s)
07:27:56.585 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.585160 7152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-2/data/info.pb
07:27:56.590 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:27:56.590 [INFO - Time-limited test] (MiniKuduCluster.java:380) Starting master server 127.6.85.253:35731
07:27:56.590 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
id {
type: MASTER
index: 1
}
}
07:27:56.590 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.589175 6487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu as pid 7152
07:27:56.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.593346 6487 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:56.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/wal
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/logs
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data/info.pb
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.6.85.253:35731
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.6.85.253
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=33219
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.253.keytab
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.6.85.253
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.6.85.212:38715
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:56.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:56.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.6.85.254:43853,127.6.85.253:35731,127.6.85.252:45547
07:27:56.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:56.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:40849/jwks.json
07:27:56.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10 with env {KRB5CCNAME=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
07:27:56.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.615139 7302 sys_catalog.cc:263] Verifying existing consensus state
07:27:56.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.620909 7302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: Bootstrap starting.
07:27:56.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.666204 7302 log.cc:824] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: Log is configured to *not* fsync() on all Append() calls
07:27:56.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.682821 7302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
07:27:56.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.683684 7302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: Bootstrap complete.
07:27:56.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.711362 7302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:56.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.714248 7302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50bb959efc9a441286c37e330a3a9dc3, State: Initialized, Role: FOLLOWER
07:27:56.716 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.715886 7302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:56.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.718962 7309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:56.720 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.719990 7309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: This master's current role is: FOLLOWER
07:27:56.720 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.720737 7302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: configured and running, proceeding with master startup.
07:27:56.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.793663 7327 catalog_manager.cc:1259] Loaded cluster ID: 225aa75c70c5475baade6fcbc26a0340
07:27:56.794 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.794341 7327 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: loading cluster ID for follower catalog manager: success
07:27:56.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.804720 7327 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: acquiring CA information for follower catalog manager: success
07:27:56.812 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.811679 7327 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:27:56.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:56 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087676, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.252@KRBTEST.COM for kudu/127.6.85.254@KRBTEST.COM
07:27:56.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.845783 7152 master_runner.cc:186] Error getting master registration for 127.6.85.253:35731: Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111),
07:27:56.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:56.880707 7152 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
07:27:57.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.123593 7305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:27:57.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.124413 7305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:27:57.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.125505 7305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:27:57.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.182798 7305 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=10
07:27:57.183 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.183133 7305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:27:57.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.183614 7305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:27:57.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.183954 7305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:27:57.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.184266 7305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:27:57.185 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.184749 7305 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:27:57.185 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.185047 7305 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:40849/jwks.json
07:27:57.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.251245 7305 master_runner.cc:386] Master server non-default flags:
07:27:57.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:27:57.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.6.85.212:38715
07:27:57.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:27:57.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:27:57.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data
07:27:57.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/wal
07:27:57.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10
07:27:57.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:27:57.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.6.85.254:43853,127.6.85.253:35731,127.6.85.252:45547
07:27:57.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/kudu_127.6.85.253.keytab
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.6.85.253
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.6.85.253:35731
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:40849/jwks.json
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=<redacted>
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data/info.pb
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.6.85.253
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=33219
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/logs
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 07 Aug 2026 06:41:47 UTC on ccycloud.quasar-byjrmo.root.comops.site
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4571901
07:27:57.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:27:57.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.252812 7305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:27:57.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.254182 7305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:27:57.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.276940 7344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:57.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.630888 7352 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
07:27:57.632 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.631683 7352 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:57.639 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.638489 7352 leader_election.cc:288] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b257a17f1bcd4e8d8bb061d0230d2529 (127.6.85.253:35731), 50bb959efc9a441286c37e330a3a9dc3 (127.6.85.252:45547)
07:27:57.669 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.669184 7067 leader_election.cc:334] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b257a17f1bcd4e8d8bb061d0230d2529 (127.6.85.253:35731): Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111)
07:27:57.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.679795 7261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90060c701cec4f34999ca0708ffc3e6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50bb959efc9a441286c37e330a3a9dc3" is_pre_election: true
07:27:57.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.681879 7261 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90060c701cec4f34999ca0708ffc3e6d in term 1.
07:27:57.685 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.684583 7071 leader_election.cc:302] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50bb959efc9a441286c37e330a3a9dc3, 90060c701cec4f34999ca0708ffc3e6d; no voters: b257a17f1bcd4e8d8bb061d0230d2529
07:27:57.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.685801 7352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 1 FOLLOWER]: Leader pre-election won for term 2
07:27:57.686 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.686142 7352 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
07:27:57.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.686496 7352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 1 FOLLOWER]: Advancing to term 2
07:27:57.697 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.696823 7352 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:57.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.698782 7352 leader_election.cc:288] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [CANDIDATE]: Term 2 election: Requested vote from peers b257a17f1bcd4e8d8bb061d0230d2529 (127.6.85.253:35731), 50bb959efc9a441286c37e330a3a9dc3 (127.6.85.252:45547)
07:27:57.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.701756 7261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "90060c701cec4f34999ca0708ffc3e6d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "50bb959efc9a441286c37e330a3a9dc3"
07:27:57.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.702534 7261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 1 FOLLOWER]: Advancing to term 2
07:27:57.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.707984 7067 leader_election.cc:334] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b257a17f1bcd4e8d8bb061d0230d2529 (127.6.85.253:35731): Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111)
07:27:57.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.712427 7261 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90060c701cec4f34999ca0708ffc3e6d in term 2.
07:27:57.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.713738 7071 leader_election.cc:302] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 50bb959efc9a441286c37e330a3a9dc3, 90060c701cec4f34999ca0708ffc3e6d; no voters: b257a17f1bcd4e8d8bb061d0230d2529
07:27:57.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.714852 7352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 2 FOLLOWER]: Leader election won for term 2
07:27:57.717 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.716929 7352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [term 2 LEADER]: Becoming Leader. State: Replica: 90060c701cec4f34999ca0708ffc3e6d, State: Running, Role: LEADER
07:27:57.718 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.717886 7352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:57.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.731958 7364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90060c701cec4f34999ca0708ffc3e6d. Latest consensus state: current_term: 2 leader_uuid: "90060c701cec4f34999ca0708ffc3e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:57.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.733218 7364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: This master's current role is: LEADER
07:27:57.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.736338 7366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
07:27:57.745 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.744757 7366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
07:27:57.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.748894 7366 catalog_manager.cc:1259] Loaded cluster ID: 225aa75c70c5475baade6fcbc26a0340
07:27:57.749 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.749292 7366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
07:27:57.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.754571 7366 catalog_manager.cc:1504] Loading token signing keys...
07:27:57.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.758703 7366 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d: Loaded TSK: 0
07:27:57.760 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.760329 7366 catalog_manager.cc:1514] Initializing in-progress tserver states...
07:27:58.190 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.190127 7261 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [term 2 FOLLOWER]: Refusing update from remote peer 90060c701cec4f34999ca0708ffc3e6d: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
07:27:58.193 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.192656 7364 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
07:27:58.209 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.208163 7368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90060c701cec4f34999ca0708ffc3e6d. Latest consensus state: current_term: 2 leader_uuid: "90060c701cec4f34999ca0708ffc3e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:58.216 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.215751 7368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: This master's current role is: FOLLOWER
07:27:58.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.230667 7368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "90060c701cec4f34999ca0708ffc3e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:58.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.231456 7368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50bb959efc9a441286c37e330a3a9dc3 [sys.catalog]: This master's current role is: FOLLOWER
07:27:58.234 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.233367 7364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "90060c701cec4f34999ca0708ffc3e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:58.234 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.234311 7364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: This master's current role is: LEADER
07:27:58.245 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:58.245182 7067 consensus_peers.cc:487] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d -> Peer b257a17f1bcd4e8d8bb061d0230d2529 (127.6.85.253:35731): Couldn't send request to peer b257a17f1bcd4e8d8bb061d0230d2529. Status: Network error: Client connection negotiation failed: client connection to 127.6.85.253:35731: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
07:27:58.687 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.277357 7343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:58.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:57.278364 7346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:27:58.688 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:57.278644 7305 server_base.cc:1034] running on GCE node
07:27:58.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.692848 7305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:27:58.696 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.696542 7305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:27:58.698 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.697988 7305 hybrid_clock.cc:648] HybridClock initialized: now 1786087678697929 us; error 76 us; skew 500 ppm
07:27:58.699 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.698881 7305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:27:58.705 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:58 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087678, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:27:58.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.706040 7305 init.cc:377] Logged in from keytab as kudu/127.6.85.253@KRBTEST.COM (short username kudu)
07:27:58.713 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.713119 7305 webserver.cc:460] Webserver started at http://127.6.85.253:33219/ using document root <none> and password file <none>
07:27:58.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.714359 7305 fs_manager.cc:359] Metadata directory not provided
07:27:58.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.714680 7305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:27:58.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.725330 7305 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.004s
07:27:58.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.731609 7387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:27:58.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.733062 7305 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
07:27:58.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.733378 7305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data,/tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/wal
07:27:58.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "b257a17f1bcd4e8d8bb061d0230d2529"
07:27:58.736 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-08-07 07:27:46 on dist-test-slave-7nm7"
07:27:58.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.736280 7305 fs_report.cc:383] FS layout report
07:27:58.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:27:58.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/wal
07:27:58.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/wal
07:27:58.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data/data
07:27:58.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:27:58.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:27:58.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:27:58.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:27:58.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:27:58.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:27:58.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:27:58.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:27:58.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:27:58.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:27:58.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:27:58.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.781644 7305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:27:58.788 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.788278 7305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:27:58.789 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.788842 7305 kserver.cc:163] Server-wide thread pool size limit: 3276
07:27:58.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.888274 7305 rpc_server.cc:307] RPC server started. Bound to: 127.6.85.253:35731
07:27:58.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.888401 7461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.85.253:35731 every 8 connection(s)
07:27:58.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.891311 7305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/master-1/data/info.pb
07:27:58.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.897699 6487 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu as pid 7305
07:27:58.898 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:27:58.899 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:27:58.900 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.6.85.252:45547(127.6.85.252:45547)] connecting to peer
07:27:58.902 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:27:58.913 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.907742 7462 sys_catalog.cc:263] Verifying existing consensus state
07:27:58.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:58.917160 7462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: Bootstrap starting.
07:27:58.923 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.252:45547(127.6.85.252:45547)] Successfully connected to peer
07:27:58.923 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:27:58.936 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:27:58.938 [DEBUG - kudu-nio-0] (SecurityContext.java:225) Refreshing Kerberos credentials...
07:27:58.938 [DEBUG - kudu-nio-0] (SecurityUtil.java:105) Using ticketCache: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5cc
07:27:58.945 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:58 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087669, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:27:58.946 [DEBUG - kudu-nio-0] (SecurityUtil.java:119) Logged in as subject: test-user@KRBTEST.COM
07:27:58.950 [ERROR - kudu-nio-0] (SecurityContext.java:249) Attempted to refresh Kerberos credentials from ticket cache but found that the new Kerberos principal test-user@KRBTEST.COM did not match the original principal test-admin@KRBTEST.COM. Ignoring.
07:27:58.951 [DEBUG - kudu-nio-1] (Negotiator.java:529) SASL mechanism GSSAPI chosen for peer 127.6.85.252
07:27:58.956 [DEBUG - kudu-nio-0] (Negotiator.java:529) SASL mechanism GSSAPI chosen for peer 127.6.85.254
07:27:58.999 [DEBUG - kudu-nio-0] (Negotiator.java:529) SASL mechanism GSSAPI chosen for peer 127.6.85.253
07:27:59.005 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:59 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.85.252@KRBTEST.COM
07:27:59.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.016254 7462 log.cc:824] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: Log is configured to *not* fsync() on all Append() calls
07:27:59.018 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: Server requires Kerberos, but this client is not authenticated (missing or expired TGT)
07:27:59.018 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: Server requires Kerberos, but this client is not authenticated (missing or expired TGT)
07:27:59.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.022183 7469 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.915206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38720 (local address 127.6.85.254:43853)
07:27:59.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.926725 (+ 11519us) server_negotiation.cc:206] Beginning negotiation
07:27:59.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.926754 (+ 29us) server_negotiation.cc:399] Waiting for connection header
07:27:59.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.935380 (+ 8626us) server_negotiation.cc:407] Connection header received
07:27:59.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.936626 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.022 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.936653 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.936887 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.937077 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.960863 (+ 23786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
07:27:59.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.963050 (+ 2187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
07:27:59.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.983051 (+ 20001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
07:27:59.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.984013 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
07:27:59.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.984521 (+ 508us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
07:27:59.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.020128 (+ 35607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38720: BlockingRecv error: failed to read from TLS socket (remote: 127.0.0.1:38720): Cannot send after transport endpoint shutdown (error 108)
07:27:59.023 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0x017a4292, L:/127.0.0.1:48978 - R:/127.6.85.252:45547] using SASL/GSSAPI
07:27:59.024 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":10838,"thread_start_us":11199,"threads_started":1}
07:27:59.040 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:27:59.043 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: Server requires Kerberos, but this client is not authenticated (missing or expired TGT)
07:27:59.044 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: Server requires Kerberos, but this client is not authenticated (missing or expired TGT)
07:27:59.044 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:27:59.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.048452 7471 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.916555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.6.85.253:35731)
07:27:59.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.935864 (+ 19309us) server_negotiation.cc:206] Beginning negotiation
07:27:59.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.935908 (+ 44us) server_negotiation.cc:399] Waiting for connection header
07:27:59.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.937561 (+ 1653us) server_negotiation.cc:407] Connection header received
07:27:59.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.944835 (+ 7274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.048 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.944899 (+ 64us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.945469 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:58.945947 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.012702 (+ 66755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
07:27:59.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.015677 (+ 2975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
07:27:59.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.040540 (+ 24863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
07:27:59.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.041673 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
07:27:59.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.042067 (+ 394us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
07:27:59.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.046702 (+ 4635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41068: BlockingRecv error: failed to read from TLS socket (remote: 127.0.0.1:41068): Cannot send after transport endpoint shutdown (error 108)
07:27:59.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":18690,"thread_start_us":19093,"threads_started":1}
07:27:59.049 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:27:59.054 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.054430 7462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
07:27:59.055 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.055480 7462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: Bootstrap complete.
07:27:59.061 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:27:59.062 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:27:59.068 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:27:59.070 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:27:59.071 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.072 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.073724 7469 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.066166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.6.85.254:43853)
07:27:59.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.066958 (+ 792us) server_negotiation.cc:206] Beginning negotiation
07:27:59.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.066984 (+ 26us) server_negotiation.cc:399] Waiting for connection header
07:27:59.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.069054 (+ 2070us) server_negotiation.cc:407] Connection header received
07:27:59.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.069560 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.069604 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.069873 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.070084 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.072727 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38736: BlockingRecv error: recv got EOF from 127.0.0.1:38736 (error 108)
07:27:59.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":359}
07:27:59.075 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:27:59.076 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.077 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.077 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:27:59.077 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:27:59.079 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.079627 7471 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.071057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41080 (local address 127.6.85.253:35731)
07:27:59.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.074818 (+ 3761us) server_negotiation.cc:206] Beginning negotiation
07:27:59.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.074845 (+ 27us) server_negotiation.cc:399] Waiting for connection header
07:27:59.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.074893 (+ 48us) server_negotiation.cc:407] Connection header received
07:27:59.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.075034 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.075065 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.075285 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.075485 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.078558 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41080: BlockingRecv error: recv got EOF from 127.0.0.1:41080 (error 108)
07:27:59.080 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":3319}
07:27:59.080 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:27:59.081 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:27:59.088 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:27:59.091 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.092 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.092 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:27:59.091 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:27:59.093 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.093591 7469 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.093 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.088010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.6.85.254:43853)
07:27:59.093 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.088659 (+ 649us) server_negotiation.cc:206] Beginning negotiation
07:27:59.093 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.088694 (+ 35us) server_negotiation.cc:399] Waiting for connection header
07:27:59.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.088968 (+ 274us) server_negotiation.cc:407] Connection header received
07:27:59.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.089649 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.089681 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.089900 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.090126 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.092659 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38752: BlockingRecv error: recv got EOF from 127.0.0.1:38752 (error 108)
07:27:59.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":261}
07:27:59.094 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.094 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.094 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.091344 7462 raft_consensus.cc:348] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:59.095 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:27:59.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.094960 7462 raft_consensus.cc:729] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b257a17f1bcd4e8d8bb061d0230d2529, State: Initialized, Role: FOLLOWER
07:27:59.096 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:27:59.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.096890 7471 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.090112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41086 (local address 127.6.85.253:35731)
07:27:59.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.090725 (+ 613us) server_negotiation.cc:206] Beginning negotiation
07:27:59.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.090752 (+ 27us) server_negotiation.cc:399] Waiting for connection header
07:27:59.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.093171 (+ 2419us) server_negotiation.cc:407] Connection header received
07:27:59.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.093327 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.093355 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.093578 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.093763 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.095775 (+ 2012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41086: BlockingRecv error: recv got EOF from 127.0.0.1:41086 (error 108)
07:27:59.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":204}
07:27:59.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.096194 7462 consensus_queue.cc:260] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } }
07:27:59.100 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:27:59.101 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:27:59.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.100898 7475 sys_catalog.cc:455] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:59.101 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.101728 7475 sys_catalog.cc:458] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: This master's current role is: FOLLOWER
07:27:59.105 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.104820 7462 sys_catalog.cc:564] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: configured and running, proceeding with master startup.
07:27:59.106 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:27:59.110 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:27:59.112 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.113 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.114 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:27:59.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.118067 7469 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.107832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.6.85.254:43853)
07:27:59.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.108479 (+ 647us) server_negotiation.cc:206] Beginning negotiation
07:27:59.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.108516 (+ 37us) server_negotiation.cc:399] Waiting for connection header
07:27:59.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.111131 (+ 2615us) server_negotiation.cc:407] Connection header received
07:27:59.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.111314 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.111344 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.111605 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.111827 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.116740 (+ 4913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38754: BlockingRecv error: recv got EOF from 127.0.0.1:38754 (error 108)
07:27:59.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":146}
07:27:59.135 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.136 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.136 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:27:59.138 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:27:59.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.138813 7471 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.105697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41098 (local address 127.6.85.253:35731)
07:27:59.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.106247 (+ 550us) server_negotiation.cc:206] Beginning negotiation
07:27:59.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.106277 (+ 30us) server_negotiation.cc:399] Waiting for connection header
07:27:59.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.107089 (+ 812us) server_negotiation.cc:407] Connection header received
07:27:59.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.108724 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.108767 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.109070 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.109305 (+ 235us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.137523 (+ 28218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41098: BlockingRecv error: recv got EOF from 127.0.0.1:41098 (error 108)
07:27:59.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":128}
07:27:59.140 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:27:59.142 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:27:59.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:59 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087674, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.254@KRBTEST.COM for kudu/127.6.85.253@KRBTEST.COM
07:27:59.147 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:27:59.150 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.150 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.151 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:27:59.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.152453 7469 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.145247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38768 (local address 127.6.85.254:43853)
07:27:59.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.145926 (+ 679us) server_negotiation.cc:206] Beginning negotiation
07:27:59.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.145964 (+ 38us) server_negotiation.cc:399] Waiting for connection header
07:27:59.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.148750 (+ 2786us) server_negotiation.cc:407] Connection header received
07:27:59.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.148901 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.148934 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.149497 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.149690 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.152 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.151675 (+ 1985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38768: BlockingRecv error: recv got EOF from 127.0.0.1:38768 (error 108)
07:27:59.153 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":147}
07:27:59.153 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:27:59.157 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.157 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.157 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:27:59.158 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:27:59.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.159945 7471 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.148807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41116 (local address 127.6.85.253:35731)
07:27:59.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.149445 (+ 638us) server_negotiation.cc:206] Beginning negotiation
07:27:59.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.149475 (+ 30us) server_negotiation.cc:399] Waiting for connection header
07:27:59.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.153522 (+ 4047us) server_negotiation.cc:407] Connection header received
07:27:59.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.155765 (+ 2243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.155802 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.156025 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.156209 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.158935 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41116: BlockingRecv error: recv got EOF from 127.0.0.1:41116 (error 108)
07:27:59.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":145}
07:27:59.167 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.166472 7429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [term 1 FOLLOWER]: Advancing to term 2
07:27:59.180 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:27:59.183 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:27:59.186 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:59 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087678, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.253@KRBTEST.COM for kudu/127.6.85.254@KRBTEST.COM
07:27:59.187 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:27:59.191 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.191 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.192 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:27:59.193 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:27:59.195 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.196 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.196 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:27:59.196 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:27:59.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.197249 7504 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.185923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38792 (local address 127.6.85.254:43853)
07:27:59.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.189501 (+ 3578us) server_negotiation.cc:206] Beginning negotiation
07:27:59.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.189537 (+ 36us) server_negotiation.cc:399] Waiting for connection header
07:27:59.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.189604 (+ 67us) server_negotiation.cc:407] Connection header received
07:27:59.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.189807 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.189839 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.190127 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.190992 (+ 865us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.196038 (+ 5046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38792: BlockingRecv error: recv got EOF from 127.0.0.1:38792 (error 108)
07:27:59.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":3030,"thread_start_us":2597,"threads_started":1}
07:27:59.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.198791 7478 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.189963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.6.85.253:35731)
07:27:59.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.190546 (+ 583us) server_negotiation.cc:206] Beginning negotiation
07:27:59.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.190572 (+ 26us) server_negotiation.cc:399] Waiting for connection header
07:27:59.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.194510 (+ 3938us) server_negotiation.cc:407] Connection header received
07:27:59.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.194709 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.194744 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.195038 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.195215 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.197363 (+ 2148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41124: BlockingRecv error: recv got EOF from 127.0.0.1:41124 (error 108)
07:27:59.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":134}
07:27:59.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.198261 7475 sys_catalog.cc:455] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90060c701cec4f34999ca0708ffc3e6d. Latest consensus state: current_term: 2 leader_uuid: "90060c701cec4f34999ca0708ffc3e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:59.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.203198 7475 sys_catalog.cc:458] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: This master's current role is: FOLLOWER
07:27:59.204 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.203969 7505 sys_catalog.cc:455] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "90060c701cec4f34999ca0708ffc3e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:59.205 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.204829 7505 sys_catalog.cc:458] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529 [sys.catalog]: This master's current role is: FOLLOWER
07:27:59.211 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.211206 7494 catalog_manager.cc:1259] Loaded cluster ID: 225aa75c70c5475baade6fcbc26a0340
07:27:59.211 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.211589 7494 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: loading cluster ID for follower catalog manager: success
07:27:59.218 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.217753 7494 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: acquiring CA information for follower catalog manager: success
07:27:59.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.224201 7494 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b257a17f1bcd4e8d8bb061d0230d2529: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:27:59.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:27:59 dist-test-slave-7nm7 krb5kdc[6512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087678, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.253@KRBTEST.COM for kudu/127.6.85.253@KRBTEST.COM
07:27:59.240 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:27:59.242 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:27:59.245 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:27:59.247 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.247 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.247 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:27:59.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.248850 7469 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.244180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38800 (local address 127.6.85.254:43853)
07:27:59.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.244796 (+ 616us) server_negotiation.cc:206] Beginning negotiation
07:27:59.249 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:27:59.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.244821 (+ 25us) server_negotiation.cc:399] Waiting for connection header
07:27:59.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.245610 (+ 789us) server_negotiation.cc:407] Connection header received
07:27:59.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.246039 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.246071 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.246343 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.246519 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.247826 (+ 1307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38800: BlockingRecv error: recv got EOF from 127.0.0.1:38800 (error 108)
07:27:59.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":194}
07:27:59.251 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.251 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.252 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:27:59.252 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:27:59.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.254871 7471 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.248784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.6.85.253:35731)
07:27:59.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.249313 (+ 529us) server_negotiation.cc:206] Beginning negotiation
07:27:59.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.249344 (+ 31us) server_negotiation.cc:399] Waiting for connection header
07:27:59.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.250328 (+ 984us) server_negotiation.cc:407] Connection header received
07:27:59.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.250454 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.250476 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.250671 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.250862 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.253890 (+ 3028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41140: BlockingRecv error: recv got EOF from 127.0.0.1:41140 (error 108)
07:27:59.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":128}
07:27:59.340 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:27:59.341 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:27:59.348 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:27:59.350 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:27:59.351 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.352 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.352 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.353 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.353 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:27:59.353 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:27:59.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.354461 7469 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.344919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38810 (local address 127.6.85.254:43853)
07:27:59.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.345474 (+ 555us) server_negotiation.cc:206] Beginning negotiation
07:27:59.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.345517 (+ 43us) server_negotiation.cc:399] Waiting for connection header
07:27:59.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.349006 (+ 3489us) server_negotiation.cc:407] Connection header received
07:27:59.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.350061 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.350104 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.350617 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.350857 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.353251 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38810: BlockingRecv error: recv got EOF from 127.0.0.1:38810 (error 108)
07:27:59.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":142}
07:27:59.355 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:27:59.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.355600 7471 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.346852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41152 (local address 127.6.85.253:35731)
07:27:59.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.347753 (+ 901us) server_negotiation.cc:206] Beginning negotiation
07:27:59.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.347786 (+ 33us) server_negotiation.cc:399] Waiting for connection header
07:27:59.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.351118 (+ 3332us) server_negotiation.cc:407] Connection header received
07:27:59.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.351652 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.351692 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.351919 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.352088 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.354461 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41152: BlockingRecv error: recv got EOF from 127.0.0.1:41152 (error 108)
07:27:59.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":105}
07:27:59.480 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:27:59.481 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:27:59.486 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:27:59.487 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.488 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.489 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:27:59.489 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:27:59.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.490298 7469 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.483345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38818 (local address 127.6.85.254:43853)
07:27:59.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.485075 (+ 1730us) server_negotiation.cc:206] Beginning negotiation
07:27:59.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.485105 (+ 30us) server_negotiation.cc:399] Waiting for connection header
07:27:59.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.486561 (+ 1456us) server_negotiation.cc:407] Connection header received
07:27:59.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.486972 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.487001 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.490 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.487263 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.487465 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.489243 (+ 1778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38818: BlockingRecv error: recv got EOF from 127.0.0.1:38818 (error 108)
07:27:59.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1287}
07:27:59.491 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:27:59.492 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:27:59.492 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:27:59.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:27:59.493534 7471 negotiation.cc:337] Failed RPC negotiation. Trace:
07:27:59.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.487805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41162 (local address 127.6.85.253:35731)
07:27:59.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.488735 (+ 930us) server_negotiation.cc:206] Beginning negotiation
07:27:59.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.488770 (+ 35us) server_negotiation.cc:399] Waiting for connection header
07:27:59.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.489413 (+ 643us) server_negotiation.cc:407] Connection header received
07:27:59.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.489834 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:27:59.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.489863 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:27:59.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.490064 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:27:59.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.490301 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
07:27:59.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:27:59.492861 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41162: BlockingRecv error: recv got EOF from 127.0.0.1:41162 (error 108)
07:27:59.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":129}
07:27:59.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.579219 7352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "90060c701cec4f34999ca0708ffc3e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90060c701cec4f34999ca0708ffc3e6d" member_type: VOTER last_known_addr { host: "127.6.85.254" port: 43853 } } peers { permanent_uuid: "b257a17f1bcd4e8d8bb061d0230d2529" member_type: VOTER last_known_addr { host: "127.6.85.253" port: 35731 } } peers { permanent_uuid: "50bb959efc9a441286c37e330a3a9dc3" member_type: VOTER last_known_addr { host: "127.6.85.252" port: 45547 } } }
07:27:59.581 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:27:59.580479 7352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90060c701cec4f34999ca0708ffc3e6d [sys.catalog]: This master's current role is: LEADER
07:28:00.000 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:00.002 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:00.006 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:00.011 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:00.012 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:00.012 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:00.012 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:00.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:00.015285 7515 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:00.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.005697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38826 (local address 127.6.85.254:43853)
07:28:00.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.010568 (+ 4871us) server_negotiation.cc:206] Beginning negotiation
07:28:00.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.010597 (+ 29us) server_negotiation.cc:399] Waiting for connection header
07:28:00.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.010654 (+ 57us) server_negotiation.cc:407] Connection header received
07:28:00.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.010920 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:00.015 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.010951 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:00.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.011285 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:00.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.012963 (+ 1678us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:00.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.013692 (+ 729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38826: BlockingRecv error: recv got EOF from 127.0.0.1:38826 (error 108)
07:28:00.016 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":4312,"thread_start_us":6269,"threads_started":1}
07:28:00.016 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:00.017 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:00.017 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:00.017 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:00.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:00.019107 7516 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:00.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.013264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41170 (local address 127.6.85.253:35731)
07:28:00.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.015376 (+ 2112us) server_negotiation.cc:206] Beginning negotiation
07:28:00.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.015398 (+ 22us) server_negotiation.cc:399] Waiting for connection header
07:28:00.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.015456 (+ 58us) server_negotiation.cc:407] Connection header received
07:28:00.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.015609 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:00.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.015637 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:00.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.015867 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:00.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.016320 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:00.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.018174 (+ 1854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41170: BlockingRecv error: recv got EOF from 127.0.0.1:41170 (error 108)
07:28:00.019 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1598,"thread_start_us":1307,"threads_started":1}
07:28:00.320 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:00.321 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:00.323 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:00.326 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:00.326 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:00.327 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:00.327 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:00.327 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:00.328 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:00.328 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:00.328 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:00.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:00.329550 7516 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:00.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.324058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41184 (local address 127.6.85.253:35731)
07:28:00.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.324676 (+ 618us) server_negotiation.cc:206] Beginning negotiation
07:28:00.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.324700 (+ 24us) server_negotiation.cc:399] Waiting for connection header
07:28:00.329 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.324766 (+ 66us) server_negotiation.cc:407] Connection header received
07:28:00.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.325771 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:00.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.325802 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:00.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.326027 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:00.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.327782 (+ 1755us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:00.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.328457 (+ 675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41184: BlockingRecv error: recv got EOF from 127.0.0.1:41184 (error 108)
07:28:00.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":140}
07:28:00.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:00.330693 7515 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:00.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.323094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38842 (local address 127.6.85.254:43853)
07:28:00.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.323732 (+ 638us) server_negotiation.cc:206] Beginning negotiation
07:28:00.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.323755 (+ 23us) server_negotiation.cc:399] Waiting for connection header
07:28:00.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.326617 (+ 2862us) server_negotiation.cc:407] Connection header received
07:28:00.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.326772 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:00.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.326798 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:00.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.327015 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:00.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.327182 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:00.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:00.329678 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38842: BlockingRecv error: recv got EOF from 127.0.0.1:38842 (error 108)
07:28:00.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":207}
07:28:01.240 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:01.241 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:01.242 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:01.243 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:01.246 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:01.246 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:01.247 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:01.247 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:01.247 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:01.247 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:01.249 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:01.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:01.249176 7520 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:01.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.242940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38858 (local address 127.6.85.254:43853)
07:28:01.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.244963 (+ 2023us) server_negotiation.cc:206] Beginning negotiation
07:28:01.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.244979 (+ 16us) server_negotiation.cc:399] Waiting for connection header
07:28:01.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.245017 (+ 38us) server_negotiation.cc:407] Connection header received
07:28:01.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.245193 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:01.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.245217 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:01.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.245449 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:01.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.245626 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:01.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.247941 (+ 2315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38858: BlockingRecv error: recv got EOF from 127.0.0.1:38858 (error 108)
07:28:01.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1628,"thread_start_us":1334,"threads_started":1}
07:28:01.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:01.250999 7521 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:01.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.244101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41198 (local address 127.6.85.253:35731)
07:28:01.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.245905 (+ 1804us) server_negotiation.cc:206] Beginning negotiation
07:28:01.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.245920 (+ 15us) server_negotiation.cc:399] Waiting for connection header
07:28:01.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.245955 (+ 35us) server_negotiation.cc:407] Connection header received
07:28:01.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.246202 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:01.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.246262 (+ 60us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:01.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.246506 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:01.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.246767 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:01.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.249809 (+ 3042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41198: BlockingRecv error: recv got EOF from 127.0.0.1:41198 (error 108)
07:28:01.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1493,"thread_start_us":1206,"threads_started":1}
07:28:01.380 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:01.381 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:01.382 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:01.383 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:01.387 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:01.387 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:01.387 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:01.388 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:01.389 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:01.389 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:01.389 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:01.389 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:01.389647 7520 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:01.390 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.382987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38866 (local address 127.6.85.254:43853)
07:28:01.390 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.385017 (+ 2030us) server_negotiation.cc:206] Beginning negotiation
07:28:01.390 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.385044 (+ 27us) server_negotiation.cc:399] Waiting for connection header
07:28:01.390 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.385093 (+ 49us) server_negotiation.cc:407] Connection header received
07:28:01.390 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.386048 (+ 955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:01.390 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.386077 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:01.390 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.386343 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:01.391 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.387972 (+ 1629us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:01.391 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.388663 (+ 691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38866: BlockingRecv error: recv got EOF from 127.0.0.1:38866 (error 108)
07:28:01.391 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1594}
07:28:01.393 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:01.393069 7521 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:01.393 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.383556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41212 (local address 127.6.85.253:35731)
07:28:01.393 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.387421 (+ 3865us) server_negotiation.cc:206] Beginning negotiation
07:28:01.393 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.387453 (+ 32us) server_negotiation.cc:399] Waiting for connection header
07:28:01.393 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.387508 (+ 55us) server_negotiation.cc:407] Connection header received
07:28:01.393 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.387664 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:01.393 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.387693 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:01.394 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.387947 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:01.394 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.391198 (+ 3251us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:01.394 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:01.392011 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41212: BlockingRecv error: recv got EOF from 127.0.0.1:41212 (error 108)
07:28:01.394 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":545}
07:28:03.620 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:03.621 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:03.623 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:03.624 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:03.628 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:03.628 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:03.629 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:03.630 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:03.631 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:03.631 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:03.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:03.631201 7528 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:03.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.623112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38874 (local address 127.6.85.254:43853)
07:28:03.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.626961 (+ 3849us) server_negotiation.cc:206] Beginning negotiation
07:28:03.631 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:03.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.626990 (+ 29us) server_negotiation.cc:399] Waiting for connection header
07:28:03.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.627060 (+ 70us) server_negotiation.cc:407] Connection header received
07:28:03.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.627310 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:03.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.627336 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:03.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.627600 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:03.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.627817 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:03.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.629895 (+ 2078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38874: BlockingRecv error: recv got EOF from 127.0.0.1:38874 (error 108)
07:28:03.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":3290,"thread_start_us":2858,"threads_started":1}
07:28:03.633 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:03.633438 7529 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:03.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.628219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41226 (local address 127.6.85.253:35731)
07:28:03.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.629621 (+ 1402us) server_negotiation.cc:206] Beginning negotiation
07:28:03.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.629644 (+ 23us) server_negotiation.cc:399] Waiting for connection header
07:28:03.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.629692 (+ 48us) server_negotiation.cc:407] Connection header received
07:28:03.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.629836 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:03.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.629860 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:03.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.630078 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:03.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.630314 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:03.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:03.632213 (+ 1899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41226: BlockingRecv error: recv got EOF from 127.0.0.1:41226 (error 108)
07:28:03.634 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":961,"thread_start_us":676,"threads_started":1}
07:28:04.271 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:28:04 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087684, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:28:04.272 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:04.272007 7060 init.cc:295] Successfully reacquired a new kerberos TGT
07:28:06.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:28:06 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087686, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:28:06.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:06.351540 7210 init.cc:295] Successfully reacquired a new kerberos TGT
07:28:06.820 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:06.821 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:06.822 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:06.823 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:06.827 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:06.828 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:06.828 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:06.829 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:06.829 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:06.829 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:06.830 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:06.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:06.830960 7538 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:06.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.825013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58108 (local address 127.6.85.253:35731)
07:28:06.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.826583 (+ 1570us) server_negotiation.cc:206] Beginning negotiation
07:28:06.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.826611 (+ 28us) server_negotiation.cc:399] Waiting for connection header
07:28:06.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.826664 (+ 53us) server_negotiation.cc:407] Connection header received
07:28:06.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.826820 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:06.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.826844 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:06.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.827081 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:06.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.828895 (+ 1814us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:06.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.829685 (+ 790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58108: BlockingRecv error: recv got EOF from 127.0.0.1:58108 (error 108)
07:28:06.831 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":909,"spinlock_wait_cycles":69504,"thread_start_us":827,"threads_started":1}
07:28:06.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:06.832083 7537 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:06.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.823283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55634 (local address 127.6.85.254:43853)
07:28:06.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.827868 (+ 4585us) server_negotiation.cc:206] Beginning negotiation
07:28:06.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.827891 (+ 23us) server_negotiation.cc:399] Waiting for connection header
07:28:06.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.827965 (+ 74us) server_negotiation.cc:407] Connection header received
07:28:06.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.828127 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:06.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.828152 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:06.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.828394 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:06.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.829701 (+ 1307us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:06.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:06.830964 (+ 1263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55634: BlockingRecv error: recv got EOF from 127.0.0.1:55634 (error 108)
07:28:06.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":4170,"thread_start_us":3841,"threads_started":1}
07:28:08.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:28:08 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087688, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:28:08.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:08.711627 7380 init.cc:295] Successfully reacquired a new kerberos TGT
07:28:10.540 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:10.541 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:10.542 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:10.546 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:10.549 [DEBUG - kudu-nio-1] (SecurityContext.java:225) Refreshing Kerberos credentials...
07:28:10.549 [DEBUG - kudu-nio-1] (SecurityUtil.java:105) Using ticketCache: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5cc
07:28:10.550 [DEBUG - kudu-nio-1] (SecurityUtil.java:122) Could not login via JAAS. Using no credentials: Unable to obtain Principal Name for authentication
07:28:10.551 [WARN - kudu-nio-1] (SecurityContext.java:234) Tried to refresh Kerberos credentials but was unable to re-login from ticket cache
07:28:10.551 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:10.551 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:10.551 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:10.551 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:10.551 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:10.551 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:10.552 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:10.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:10.554275 7548 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:10.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.542898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55638 (local address 127.6.85.254:43853)
07:28:10.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.546539 (+ 3641us) server_negotiation.cc:206] Beginning negotiation
07:28:10.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.546561 (+ 22us) server_negotiation.cc:399] Waiting for connection header
07:28:10.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.547585 (+ 1024us) server_negotiation.cc:407] Connection header received
07:28:10.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.547744 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:10.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.547770 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:10.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.548779 (+ 1009us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:10.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.548967 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:10.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.552457 (+ 3490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55638: BlockingRecv error: recv got EOF from 127.0.0.1:55638 (error 108)
07:28:10.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":3180,"thread_start_us":2833,"threads_started":1}
07:28:10.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:10.555186 7549 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:10.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.546569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58124 (local address 127.6.85.253:35731)
07:28:10.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.549053 (+ 2484us) server_negotiation.cc:206] Beginning negotiation
07:28:10.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.549081 (+ 28us) server_negotiation.cc:399] Waiting for connection header
07:28:10.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.549144 (+ 63us) server_negotiation.cc:407] Connection header received
07:28:10.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.549348 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:10.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.549371 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:10.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.549626 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:10.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.550006 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:10.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:10.553562 (+ 3556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58124: BlockingRecv error: recv got EOF from 127.0.0.1:58124 (error 108)
07:28:10.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":2043,"thread_start_us":1697,"threads_started":1}
07:28:12.980 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:12.981 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:12.982 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:12.983 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:12.988 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:12.989 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:12.989 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:12.989 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:12.989 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:12.989 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:12.990 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:12.991721 7557 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.984114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58130 (local address 127.6.85.253:35731)
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.987457 (+ 3343us) server_negotiation.cc:206] Beginning negotiation
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.987479 (+ 22us) server_negotiation.cc:399] Waiting for connection header
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.987526 (+ 47us) server_negotiation.cc:407] Connection header received
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.987719 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.987749 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.988011 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.988638 (+ 627us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.990412 (+ 1774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58130: BlockingRecv error: recv got EOF from 127.0.0.1:58130 (error 108)
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":2904,"thread_start_us":2593,"threads_started":1}
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:12.991753 7554 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.983689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55640 (local address 127.6.85.254:43853)
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.987051 (+ 3362us) server_negotiation.cc:206] Beginning negotiation
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.987091 (+ 40us) server_negotiation.cc:399] Waiting for connection header
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.987144 (+ 53us) server_negotiation.cc:407] Connection header received
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.987327 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.987342 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.987562 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.987892 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:12.990060 (+ 2168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55640: BlockingRecv error: recv got EOF from 127.0.0.1:55640 (error 108)
07:28:12.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":2697,"thread_start_us":2148,"threads_started":1}
07:28:14.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:28:14 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087694, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:28:14.275 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:14.275079 7060 init.cc:295] Successfully reacquired a new kerberos TGT
07:28:16.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:28:16 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087696, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:28:16.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:16.354152 7210 init.cc:295] Successfully reacquired a new kerberos TGT
07:28:16.460 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:16.462 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:16.462 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:16.462 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:16.465 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:16.465 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:16.466 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:16.468 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:16.469 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:16.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:16.468885 7561 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:16.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.462894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50048 (local address 127.6.85.254:43853)
07:28:16.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.464248 (+ 1354us) server_negotiation.cc:206] Beginning negotiation
07:28:16.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.464275 (+ 27us) server_negotiation.cc:399] Waiting for connection header
07:28:16.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.464320 (+ 45us) server_negotiation.cc:407] Connection header received
07:28:16.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.464515 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:16.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.464540 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:16.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.464802 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:16.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.467183 (+ 2381us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:16.469 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:16.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.467902 (+ 719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50048: BlockingRecv error: recv got EOF from 127.0.0.1:50048 (error 108)
07:28:16.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":959,"thread_start_us":1038,"threads_started":1}
07:28:16.470 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:16.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:16.471381 7562 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:16.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.463716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36832 (local address 127.6.85.253:35731)
07:28:16.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.467586 (+ 3870us) server_negotiation.cc:206] Beginning negotiation
07:28:16.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.467608 (+ 22us) server_negotiation.cc:399] Waiting for connection header
07:28:16.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.467656 (+ 48us) server_negotiation.cc:407] Connection header received
07:28:16.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.467798 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:16.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.467828 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:16.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.468048 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:16.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.468245 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:16.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:16.470482 (+ 2237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36832: BlockingRecv error: recv got EOF from 127.0.0.1:36832 (error 108)
07:28:16.471 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":3444,"thread_start_us":3137,"threads_started":1}
07:28:18.713 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:28:18 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087698, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:28:18.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:18.714381 7380 init.cc:295] Successfully reacquired a new kerberos TGT
07:28:19.220 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:19.221 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:19.222 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:19.222 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:19.228 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:19.228 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:19.228 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:19.229 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:19.229 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:19.229 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:19.229 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:19.230728 7575 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.223302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36846 (local address 127.6.85.253:35731)
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.227328 (+ 4026us) server_negotiation.cc:206] Beginning negotiation
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.227350 (+ 22us) server_negotiation.cc:399] Waiting for connection header
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.227400 (+ 50us) server_negotiation.cc:407] Connection header received
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.227575 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.227590 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.227766 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.227982 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.229506 (+ 1524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36846: BlockingRecv error: recv got EOF from 127.0.0.1:36846 (error 108)
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":3448,"thread_start_us":3044,"threads_started":1}
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:19.230844 7574 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:19.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.222776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50054 (local address 127.6.85.254:43853)
07:28:19.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.227748 (+ 4972us) server_negotiation.cc:206] Beginning negotiation
07:28:19.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.227765 (+ 17us) server_negotiation.cc:399] Waiting for connection header
07:28:19.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.227806 (+ 41us) server_negotiation.cc:407] Connection header received
07:28:19.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.227930 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:19.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.227948 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:19.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.228158 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:19.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.228369 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:19.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:19.230008 (+ 1639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50054: BlockingRecv error: recv got EOF from 127.0.0.1:50054 (error 108)
07:28:19.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":4519,"thread_start_us":4216,"threads_started":1}
07:28:20.220 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:20.221 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:20.221 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:20.222 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:20.227 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:20.227 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:20.227 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:20.227 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:20.227 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:20.227 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:20.228 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:20.229773 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.222775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36860 (local address 127.6.85.253:35731)
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.226188 (+ 3413us) server_negotiation.cc:206] Beginning negotiation
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.226205 (+ 17us) server_negotiation.cc:399] Waiting for connection header
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.226329 (+ 124us) server_negotiation.cc:407] Connection header received
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.226485 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.226514 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.226769 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.226988 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.228762 (+ 1774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36860: BlockingRecv error: recv got EOF from 127.0.0.1:36860 (error 108)
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":3069,"thread_start_us":2778,"threads_started":1}
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:20.229754 7583 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.222828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50058 (local address 127.6.85.254:43853)
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.225898 (+ 3070us) server_negotiation.cc:206] Beginning negotiation
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.225914 (+ 16us) server_negotiation.cc:399] Waiting for connection header
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.225954 (+ 40us) server_negotiation.cc:407] Connection header received
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.226070 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.226086 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.226276 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.226476 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.228444 (+ 1968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50058: BlockingRecv error: recv got EOF from 127.0.0.1:50058 (error 108)
07:28:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":2542,"thread_start_us":2194,"threads_started":1}
07:28:20.640 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:20.641 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:20.642 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:20.642 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:20.645 [DEBUG - kudu-nio-0] (SecurityContext.java:225) Refreshing Kerberos credentials...
07:28:20.645 [DEBUG - kudu-nio-0] (SecurityUtil.java:105) Using ticketCache: /tmp/dist-test-taskO_PLPU/test-tmp/mini-kudu-cluster4799516192733452609/krb5kdc/krb5cc
07:28:20.646 [DEBUG - kudu-nio-0] (SecurityUtil.java:122) Could not login via JAAS. Using no credentials: Unable to obtain Principal Name for authentication
07:28:20.646 [WARN - kudu-nio-0] (SecurityContext.java:234) Tried to refresh Kerberos credentials but was unable to re-login from ticket cache
07:28:20.646 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:20.646 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:20.647 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:20.647 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:20.647 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:20.647 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:20.648 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:20.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:20.648644 7583 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:20.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.643663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.6.85.254:43853)
07:28:20.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644199 (+ 536us) server_negotiation.cc:206] Beginning negotiation
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644225 (+ 26us) server_negotiation.cc:399] Waiting for connection header
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644269 (+ 44us) server_negotiation.cc:407] Connection header received
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644396 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644420 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644648 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644829 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.647710 (+ 2881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50060: BlockingRecv error: recv got EOF from 127.0.0.1:50060 (error 108)
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":125}
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:20.649106 7582 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.643552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36872 (local address 127.6.85.253:35731)
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644084 (+ 532us) server_negotiation.cc:206] Beginning negotiation
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644111 (+ 27us) server_negotiation.cc:399] Waiting for connection header
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644162 (+ 51us) server_negotiation.cc:407] Connection header received
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644308 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644334 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644595 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.644826 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:20.648337 (+ 3511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36872: BlockingRecv error: recv got EOF from 127.0.0.1:36872 (error 108)
07:28:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":106}
07:28:21.580 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:21.581 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:21.581 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:21.581 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:21.586 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:21.587 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:21.587 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:21.588 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:21.589 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:21.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:21.589108 7587 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:21.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.583114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.6.85.254:43853)
07:28:21.589 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:21.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.585455 (+ 2341us) server_negotiation.cc:206] Beginning negotiation
07:28:21.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.585485 (+ 30us) server_negotiation.cc:399] Waiting for connection header
07:28:21.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.585534 (+ 49us) server_negotiation.cc:407] Connection header received
07:28:21.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.585676 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:21.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.585701 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:21.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.585927 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:21.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.587135 (+ 1208us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:21.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.587949 (+ 814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50066: BlockingRecv error: recv got EOF from 127.0.0.1:50066 (error 108)
07:28:21.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1911,"thread_start_us":3106,"threads_started":1}
07:28:21.589 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:21.591274 7589 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.583266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36874 (local address 127.6.85.253:35731)
07:28:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.587396 (+ 4130us) server_negotiation.cc:206] Beginning negotiation
07:28:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.587419 (+ 23us) server_negotiation.cc:399] Waiting for connection header
07:28:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.587573 (+ 154us) server_negotiation.cc:407] Connection header received
07:28:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.587783 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.587820 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.588056 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.589186 (+ 1130us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:21.590396 (+ 1210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36874: BlockingRecv error: recv got EOF from 127.0.0.1:36874 (error 108)
07:28:21.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":3706,"thread_start_us":3491,"threads_started":1}
07:28:23.360 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:23.361 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:23.362 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:23.362 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:23.366 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:23.367 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:23.367 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:23.367 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:23.367 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:23.367 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:23.368 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:23.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:23.368964 7591 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:23.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.363923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36888 (local address 127.6.85.253:35731)
07:28:23.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.365452 (+ 1529us) server_negotiation.cc:206] Beginning negotiation
07:28:23.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.365467 (+ 15us) server_negotiation.cc:399] Waiting for connection header
07:28:23.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.365503 (+ 36us) server_negotiation.cc:407] Connection header received
07:28:23.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.365613 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:23.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.365629 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:23.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.365792 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:23.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.365936 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:23.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.367818 (+ 1882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36888: BlockingRecv error: recv got EOF from 127.0.0.1:36888 (error 108)
07:28:23.369 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1056,"thread_start_us":594,"threads_started":1}
07:28:23.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:23.370380 7592 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:23.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.363883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50072 (local address 127.6.85.254:43853)
07:28:23.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.365783 (+ 1900us) server_negotiation.cc:206] Beginning negotiation
07:28:23.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.365804 (+ 21us) server_negotiation.cc:399] Waiting for connection header
07:28:23.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.365853 (+ 49us) server_negotiation.cc:407] Connection header received
07:28:23.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.366194 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:23.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.366257 (+ 63us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:23.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.366650 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:23.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.366871 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:23.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:23.368566 (+ 1695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50072: BlockingRecv error: recv got EOF from 127.0.0.1:50072 (error 108)
07:28:23.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":849,"thread_start_us":497,"threads_started":1}
07:28:24.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:28:24 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087704, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:28:24.277 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:24.277623 7060 init.cc:295] Successfully reacquired a new kerberos TGT
07:28:25.259 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:25.260 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:25.261 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:25.262 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:25.265 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:25.265 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:25.265 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:25.265 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:25.265 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:25.266 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:25.266 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:25.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:25.267351 7600 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:25.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.262484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36568 (local address 127.6.85.253:35731)
07:28:25.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.263805 (+ 1321us) server_negotiation.cc:206] Beginning negotiation
07:28:25.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.263827 (+ 22us) server_negotiation.cc:399] Waiting for connection header
07:28:25.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.263876 (+ 49us) server_negotiation.cc:407] Connection header received
07:28:25.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.264123 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:25.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.264152 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:25.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.264404 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:25.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.264810 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:25.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.266089 (+ 1279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36568: BlockingRecv error: recv got EOF from 127.0.0.1:36568 (error 108)
07:28:25.267 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":901,"thread_start_us":556,"threads_started":1}
07:28:25.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:25.267858 7598 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:25.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.261934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40788 (local address 127.6.85.254:43853)
07:28:25.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.264401 (+ 2467us) server_negotiation.cc:206] Beginning negotiation
07:28:25.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.264434 (+ 33us) server_negotiation.cc:399] Waiting for connection header
07:28:25.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.264486 (+ 52us) server_negotiation.cc:407] Connection header received
07:28:25.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.264633 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:25.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.264658 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:25.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.264942 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:25.268 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.265278 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:25.269 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:25.266771 (+ 1493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40788: BlockingRecv error: recv got EOF from 127.0.0.1:40788 (error 108)
07:28:25.269 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1958,"thread_start_us":2258,"threads_started":1}
07:28:26.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Aug 07 07:28:26 dist-test-slave-7nm7 krb5kdc[6512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786087706, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.85.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:28:26.357 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:26.356822 7210 init.cc:295] Successfully reacquired a new kerberos TGT
07:28:28.180 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.254:43853(127.6.85.254:43853)] connecting to peer
07:28:28.181 [DEBUG - pool-44-thread-1] (Connection.java:748) [peer master-127.6.85.253:35731(127.6.85.253:35731)] connecting to peer
07:28:28.181 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.6.85.254:43853(127.6.85.254:43853)] Successfully connected to peer
07:28:28.181 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.6.85.253:35731(127.6.85.253:35731)] Successfully connected to peer
07:28:28.186 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.6.85.253:35731(127.6.85.253:35731)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:28.186 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.254:43853(127.6.85.254:43853)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:28.186 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.253:35731: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:28.187 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.6.85.254:43853: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:28:28.187 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.6.85.253:35731(127.6.85.253:35731)] handling channelInactive
07:28:28.187 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731; will retry
07:28:28.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:28.188828 7610 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:28.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.182616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.6.85.253:35731)
07:28:28.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.185235 (+ 2619us) server_negotiation.cc:206] Beginning negotiation
07:28:28.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.185264 (+ 29us) server_negotiation.cc:399] Waiting for connection header
07:28:28.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.185338 (+ 74us) server_negotiation.cc:407] Connection header received
07:28:28.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.185569 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:28.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.185597 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:28.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.185839 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:28.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.186082 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:28.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.187434 (+ 1352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36580: BlockingRecv error: recv got EOF from 127.0.0.1:36580 (error 108)
07:28:28.189 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":2097,"thread_start_us":1742,"threads_started":1}
07:28:28.192 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:1887) Cannot continue with RPC because of: Timed out: cannot complete before timeout: KuduRpc(method=ListTabletServers, tablet=null, attempt=25, TimeoutTracker(timeout=30000, elapsed=29288), Traces: [0ms] refreshing cache from master, [0ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [1ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [2ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [119ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Configuration error: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [136ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [144ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Configuration error: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [146ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but this client is not authenticated (missing or expired TGT),org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [161ms] refreshing cache from master, [162ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [163ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [163ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [169ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [172ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [177ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [178ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [181ms] refreshing cache from master, [181ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [182ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [182ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [189ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [192ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [195ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [196ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [200ms] refreshing cache from master, [201ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [202ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [202ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [211ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [214ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [236ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [237ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [241ms] refreshing cache from master, [241ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [242ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [242ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [248ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [251ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [258ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [258ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [280ms] refreshing cache from master, [281ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [283ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [283ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [288ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [292ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [296ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [297ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [341ms] refreshing cache from master, [341ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [342ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [343ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [347ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [348ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [352ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [353ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [440ms] refreshing cache from master, [441ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [449ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [453ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [453ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [454ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [580ms] refreshing cache from master, [581ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [581ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [582ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [587ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [589ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [591ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [593ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1101ms] refreshing cache from master, [1101ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [1102ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [1102ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [1108ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [1113ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1117ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1118ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1420ms] refreshing cache from master, [1420ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [1421ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [1422ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [1426ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [1428ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1428ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1429ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2341ms] refreshing cache from master, [2341ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [2342ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [2342ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [2346ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [2347ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2348ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2349ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2480ms] refreshing cache from master, [2481ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [2482ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [2482ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [2486ms] too many traces: truncated at 100 traces, deferred=Deferred@276826420(state=PENDING, result=null, callback=(continuation of Deferred@995042789 after retry RPC after error@927238483) -> (continuation of Deferred@1686896755 after retry RPC after error@1960148095) -> (continuation of Deferred@913776397 after retry RPC after error@627709569) -> (continuation of Deferred@560010695 after retry RPC after error@2122883856) -> (continuation of Deferred@1879655611 after retry RPC after error@86153765) -> (continuation of Deferred@751073387 after retry RPC after error@401592000) -> (continuation of Deferred@762932250 after retry RPC after error@1016801314) -> (continuation of Deferred@865274106 after retry RPC after error@1524849030) -> (continuation of Deferred@1008527228 after retry RPC after error@1495466725) -> (continuation of Deferred@1918637539 after retry RPC after error@850615623) -> (continuation of Deferred@1766812522 after retry RPC after error@683309196) -> (continuation of Deferred@987468661 after retry RPC after error@313174028) -> (continuation of Deferred@11306248 after retry RPC after error@2074731401) -> (continuation of Deferred@487561138 after retry RPC after error@1520608983) -> (continuation of Deferred@1044577358 after retry RPC after error@180614564) -> (continuation of Deferred@1804620105 after retry RPC after error@617338610) -> (continuation of Deferred@230878889 after retry RPC after error@406410336) -> (continuation of Deferred@1909081143 after retry RPC after error@189558970) -> (continuation of Deferred@299987164 after retry RPC after error@1126804692) -> (continuation of Deferred@1625400186 after retry RPC after error@146855548) -> (continuation of Deferred@305995816 after retry RPC after error@486306231) -> (continuation of Deferred@1911290657 after retry RPC after error@2140577878) -> (continuation of Deferred@80864236 after retry RPC after error@2069842696) -> (continuation of Deferred@1438877105 after retry RPC after error@522001231), errback=(continuation of Deferred@995042789 after retry RPC after error@927238483) -> (continuation of Deferred@1686896755 after retry RPC after error@1960148095) -> (continuation of Deferred@913776397 after retry RPC after error@627709569) -> (continuation of Deferred@560010695 after retry RPC after error@2122883856) -> (continuation of Deferred@1879655611 after retry RPC after error@86153765) -> (continuation of Deferred@751073387 after retry RPC after error@401592000) -> (continuation of Deferred@762932250 after retry RPC after error@1016801314) -> (continuation of Deferred@865274106 after retry RPC after error@1524849030) -> (continuation of Deferred@1008527228 after retry RPC after error@1495466725) -> (continuation of Deferred@1918637539 after retry RPC after error@850615623) -> (continuation of Deferred@1766812522 after retry RPC after error@683309196) -> (continuation of Deferred@987468661 after retry RPC after error@313174028) -> (continuation of Deferred@11306248 after retry RPC after error@2074731401) -> (continuation of Deferred@487561138 after retry RPC after error@1520608983) -> (continuation of Deferred@1044577358 after retry RPC after error@180614564) -> (continuation of Deferred@1804620105 after retry RPC after error@617338610) -> (continuation of Deferred@230878889 after retry RPC after error@406410336) -> (continuation of Deferred@1909081143 after retry RPC after error@189558970) -> (continuation of Deferred@299987164 after retry RPC after error@1126804692) -> (continuation of Deferred@1625400186 after retry RPC after error@146855548) -> (continuation of Deferred@305995816 after retry RPC after error@486306231) -> (continuation of Deferred@1911290657 after retry RPC after error@2140577878) -> (continuation of Deferred@80864236 after retry RPC after error@2069842696) -> (continuation of Deferred@1438877105 after retry RPC after error@522001231)))
07:28:28.194 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.254:43853(127.6.85.254:43853)] handling channelInactive
07:28:28.195 [DEBUG - main] (Connection.java:575) [peer master-127.6.85.252:45547(127.6.85.252:45547)] disconnecting while in state READY
07:28:28.195 [DEBUG - main] (Connection.java:575) [peer master-127.6.85.253:35731(127.6.85.253:35731)] disconnecting while in state TERMINATED
07:28:28.195 [DEBUG - main] (Connection.java:575) [peer master-127.6.85.254:43853(127.6.85.254:43853)] disconnecting while in state TERMINATED
07:28:28.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260807 07:28:28.196738 7611 negotiation.cc:337] Failed RPC negotiation. Trace:
07:28:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.183091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40800 (local address 127.6.85.254:43853)
07:28:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.185704 (+ 2613us) server_negotiation.cc:206] Beginning negotiation
07:28:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.185726 (+ 22us) server_negotiation.cc:399] Waiting for connection header
07:28:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.185773 (+ 47us) server_negotiation.cc:407] Connection header received
07:28:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.185913 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:28:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.185931 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:28:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.186164 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:28:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.187612 (+ 1448us) server_negotiation.cc:226] Negotiated authn=SASL
07:28:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0807 07:28:28.195597 (+ 7985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40800: BlockingRecv error: recv got EOF from 127.0.0.1:40800 (error 108)
07:28:28.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":2171,"thread_start_us":1866,"threads_started":1}
07:28:28.198 [DEBUG - kudu-nio-1] (AsyncKuduClient.java:2704) Releasing all remaining resources
07:28:28.201 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.6.85.252:45547(127.6.85.252:45547)] handling channelInactive
07:28:28.201 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.6.85.252:45547(127.6.85.252:45547)] cleaning up while in state READY due to: connection closed
07:28:28.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:28.201261 6487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu with pid 7040
07:28:28.270 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:28.263912 6487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu with pid 7305
07:28:28.303 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:28.302945 6487 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu with pid 7152
07:28:29.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-07T07:28:29Z chronyd exiting
07:28:30.623 [DEBUG - main] (AsyncKuduClient.java:2704) Releasing all remaining resources
07:28:30.631 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:30.625291 5329 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu with pid 5774
07:28:30.668 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:30.668143 5329 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu with pid 6043
07:28:30.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:30.712195 5329 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu with pid 6261
07:28:30.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:30.751040 5329 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu with pid 5347
07:28:30.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:30.797420 5329 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu with pid 5446
07:28:30.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260807 07:28:30.844287 5329 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO_PLPU/build/tsan/bin/kudu with pid 5579
07:28:32.070 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-08-07T07:28:32Z chronyd exiting
07:28:33.120 [ERROR - main] (RetryRule.java:206) org.apache.kudu.client.TestSecurity.testDoNotSwitchPrincipalsInExistingClient: failed attempt 1
org.apache.kudu.client.NonRecoverableException: cannot complete before timeout: KuduRpc(method=ListTabletServers, tablet=null, attempt=25, TimeoutTracker(timeout=30000, elapsed=29288), Traces: [0ms] refreshing cache from master, [0ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [1ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [2ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [119ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Configuration error: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [136ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [144ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Configuration error: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [146ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but this client is not authenticated (missing or expired TGT),org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [161ms] refreshing cache from master, [162ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [163ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [163ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [169ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [172ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [177ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [178ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [181ms] refreshing cache from master, [181ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [182ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [182ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [189ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [192ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [195ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [196ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [200ms] refreshing cache from master, [201ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [202ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [202ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [211ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [214ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [236ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [237ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [241ms] refreshing cache from master, [241ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [242ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [242ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [248ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [251ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [258ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [258ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [280ms] refreshing cache from master, [281ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [283ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [283ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [288ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [292ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [296ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [297ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [341ms] refreshing cache from master, [341ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [342ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [343ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [347ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [348ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [352ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [353ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [440ms] refreshing cache from master, [441ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [449ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [453ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [453ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [454ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [580ms] refreshing cache from master, [581ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [581ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [582ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [587ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [589ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [591ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [593ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1101ms] refreshing cache from master, [1101ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [1102ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [1102ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [1108ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [1113ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1117ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1118ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1420ms] refreshing cache from master, [1420ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [1421ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [1422ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [1426ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [1428ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1428ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1429ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2341ms] refreshing cache from master, [2341ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [2342ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [2342ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [2346ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [2347ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2348ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2349ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2480ms] refreshing cache from master, [2481ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [2482ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [2482ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [2486ms] too many traces: truncated at 100 traces, deferred=Deferred@276826420(state=PENDING, result=null, callback=(continuation of Deferred@995042789 after retry RPC after error@927238483) -> (continuation of Deferred@1686896755 after retry RPC after error@1960148095) -> (continuation of Deferred@913776397 after retry RPC after error@627709569) -> (continuation of Deferred@560010695 after retry RPC after error@2122883856) -> (continuation of Deferred@1879655611 after retry RPC after error@86153765) -> (continuation of Deferred@751073387 after retry RPC after error@401592000) -> (continuation of Deferred@762932250 after retry RPC after error@1016801314) -> (continuation of Deferred@865274106 after retry RPC after error@1524849030) -> (continuation of Deferred@1008527228 after retry RPC after error@1495466725) -> (continuation of Deferred@1918637539 after retry RPC after error@850615623) -> (continuation of Deferred@1766812522 after retry RPC after error@683309196) -> (continuation of Deferred@987468661 after retry RPC after error@313174028) -> (continuation of Deferred@11306248 after retry RPC after error@2074731401) -> (continuation of Deferred@487561138 after retry RPC after error@1520608983) -> (continuation of Deferred@1044577358 after retry RPC after error@180614564) -> (continuation of Deferred@1804620105 after retry RPC after error@617338610) -> (continuation of Deferred@230878889 after retry RPC after error@406410336) -> (continuation of Deferred@1909081143 after retry RPC after error@189558970) -> (continuation of Deferred@299987164 after retry RPC after error@1126804692) -> (continuation of Deferred@1625400186 after retry RPC after error@146855548) -> (continuation of Deferred@305995816 after retry RPC after error@486306231) -> (continuation of Deferred@1911290657 after retry RPC after error@2140577878) -> (continuation of Deferred@80864236 after retry RPC after error@2069842696) -> (continuation of Deferred@1438877105 after retry RPC after error@522001231), errback=(continuation of Deferred@995042789 after retry RPC after error@927238483) -> (continuation of Deferred@1686896755 after retry RPC after error@1960148095) -> (continuation of Deferred@913776397 after retry RPC after error@627709569) -> (continuation of Deferred@560010695 after retry RPC after error@2122883856) -> (continuation of Deferred@1879655611 after retry RPC after error@86153765) -> (continuation of Deferred@751073387 after retry RPC after error@401592000) -> (continuation of Deferred@762932250 after retry RPC after error@1016801314) -> (continuation of Deferred@865274106 after retry RPC after error@1524849030) -> (continuation of Deferred@1008527228 after retry RPC after error@1495466725) -> (continuation of Deferred@1918637539 after retry RPC after error@850615623) -> (continuation of Deferred@1766812522 after retry RPC after error@683309196) -> (continuation of Deferred@987468661 after retry RPC after error@313174028) -> (continuation of Deferred@11306248 after retry RPC after error@2074731401) -> (continuation of Deferred@487561138 after retry RPC after error@1520608983) -> (continuation of Deferred@1044577358 after retry RPC after error@180614564) -> (continuation of Deferred@1804620105 after retry RPC after error@617338610) -> (continuation of Deferred@230878889 after retry RPC after error@406410336) -> (continuation of Deferred@1909081143 after retry RPC after error@189558970) -> (continuation of Deferred@299987164 after retry RPC after error@1126804692) -> (continuation of Deferred@1625400186 after retry RPC after error@146855548) -> (continuation of Deferred@305995816 after retry RPC after error@486306231) -> (continuation of Deferred@1911290657 after retry RPC after error@2140577878) -> (continuation of Deferred@80864236 after retry RPC after error@2069842696) -> (continuation of Deferred@1438877105 after retry RPC after error@522001231)))
at org.apache.kudu.client.KuduException.transformException(KuduException.java:110) ~[main/:?]
at org.apache.kudu.client.KuduClient.joinAndHandleException(KuduClient.java:564) ~[main/:?]
at org.apache.kudu.client.KuduClient.listTabletServers(KuduClient.java:263) ~[main/:?]
at org.apache.kudu.client.TestSecurity$3.get(TestSecurity.java:386) ~[test/:?]
at org.apache.kudu.test.junit.AssertHelpers.assertEventuallyTrue(AssertHelpers.java:40) ~[kudu-test-utils-1.18.0-SNAPSHOT-unshaded.jar:1.18.0-SNAPSHOT]
at org.apache.kudu.client.TestSecurity.assertEventualAuthenticationFailure(TestSecurity.java:378) ~[test/:?]
at org.apache.kudu.client.TestSecurity.testDoNotSwitchPrincipalsInExistingClient(TestSecurity.java:366) ~[test/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_362]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_362]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_362]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_362]
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) ~[junit-4.13.2.jar:4.13.2]
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) ~[junit-4.13.2.jar:4.13.2]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_362]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_362]
Suppressed: org.apache.kudu.client.KuduException$OriginalException: Original asynchronous stack trace
at org.apache.kudu.client.AsyncKuduClient.tooManyAttemptsOrTimeout(AsyncKuduClient.java:1889) ~[main/:?]
at org.apache.kudu.client.AsyncKuduClient.delayedSendRpcToTablet(AsyncKuduClient.java:2337) ~[main/:?]
at org.apache.kudu.client.AsyncKuduClient.access$1500(AsyncKuduClient.java:299) ~[main/:?]
at org.apache.kudu.client.AsyncKuduClient$RetryRpcErrback.call(AsyncKuduClient.java:1520) ~[main/:?]
at org.apache.kudu.client.AsyncKuduClient$RetryRpcErrback.call(AsyncKuduClient.java:1510) ~[main/:?]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002) ~[async-1.4.1.jar:?]
at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:313) ~[main/:?]
at org.apache.kudu.client.ConnectToCluster.access$000(ConnectToCluster.java:51) ~[main/:?]
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:387) ~[main/:?]
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:376) ~[main/:?]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002) ~[async-1.4.1.jar:?]
at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:277) ~[main/:?]
at org.apache.kudu.client.KuduRpc.errback(KuduRpc.java:331) ~[main/:?]
at org.apache.kudu.client.RpcProxy.responseReceived(RpcProxy.java:256) ~[main/:?]
at org.apache.kudu.client.RpcProxy.access$000(RpcProxy.java:64) ~[main/:?]
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:158) ~[main/:?]
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:154) ~[main/:?]
at org.apache.kudu.client.Connection.cleanup(Connection.java:737) ~[main/:?]
at org.apache.kudu.client.Connection.exceptionCaught(Connection.java:455) ~[main/:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:447) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.94.Final.jar:4.1.94.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.94.Final.jar:4.1.94.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_362]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_362]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_362]
Caused by: org.apache.kudu.client.NoLeaderFoundException: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:311) ~[main/:?]
at org.apache.kudu.client.ConnectToCluster.access$000(ConnectToCluster.java:51) ~[main/:?]
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:387) ~[main/:?]
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:376) ~[main/:?]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002) ~[async-1.4.1.jar:?]
at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:277) ~[main/:?]
at org.apache.kudu.client.KuduRpc.errback(KuduRpc.java:331) ~[main/:?]
at org.apache.kudu.client.RpcProxy.responseReceived(RpcProxy.java:256) ~[main/:?]
at org.apache.kudu.client.RpcProxy.access$000(RpcProxy.java:64) ~[main/:?]
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:158) ~[main/:?]
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:154) ~[main/:?]
at org.apache.kudu.client.Connection.cleanup(Connection.java:737) ~[main/:?]
at org.apache.kudu.client.Connection.exceptionCaught(Connection.java:455) ~[main/:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:447) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.94.Final.jar:4.1.94.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.94.Final.jar:4.1.94.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_362]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_362]
... 1 more
Caused by: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
at org.apache.kudu.client.Negotiator.chooseAndInitializeSaslMech(Negotiator.java:565) ~[main/:?]
at org.apache.kudu.client.Negotiator.handleNegotiateResponse(Negotiator.java:433) ~[main/:?]
at org.apache.kudu.client.Negotiator.channelRead0(Negotiator.java:367) ~[main/:?]
at org.apache.kudu.client.Negotiator.channelRead0(Negotiator.java:94) ~[main/:?]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.94.Final.jar:4.1.94.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.94.Final.jar:4.1.94.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_362]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_362]
... 1 more
org.apache.kudu.client.NonRecoverableException: cannot complete before timeout: KuduRpc(method=ListTabletServers, tablet=null, attempt=25, TimeoutTracker(timeout=30000, elapsed=29288), Traces: [0ms] refreshing cache from master, [0ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [1ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [2ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [119ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Configuration error: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [136ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [144ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Configuration error: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [146ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but this client is not authenticated (missing or expired TGT),org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [161ms] refreshing cache from master, [162ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [163ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [163ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [169ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [172ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [177ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [178ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [181ms] refreshing cache from master, [181ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [182ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [182ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [189ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [192ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [195ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [196ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [200ms] refreshing cache from master, [201ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [202ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [202ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [211ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [214ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [236ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [237ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [241ms] refreshing cache from master, [241ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [242ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [242ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [248ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [251ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [258ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [258ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [280ms] refreshing cache from master, [281ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [283ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [283ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [288ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [292ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [296ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [297ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [341ms] refreshing cache from master, [341ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [342ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [343ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [347ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [348ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [352ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [353ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [440ms] refreshing cache from master, [441ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [449ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [453ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [453ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [454ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [580ms] refreshing cache from master, [581ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [581ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [582ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [587ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [589ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [591ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [593ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1101ms] refreshing cache from master, [1101ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [1102ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [1102ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [1108ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [1113ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1117ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1118ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1420ms] refreshing cache from master, [1420ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [1421ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [1422ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [1426ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [1428ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1428ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1429ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2341ms] refreshing cache from master, [2341ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [2342ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [2342ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [2346ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.252:45547: OK, [2347ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.254:43853: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2348ms] Sub RPC ConnectToMaster: received response from server master-127.6.85.253:35731: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2349ms] delaying RPC due to: Service unavailable: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2480ms] refreshing cache from master, [2481ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.254:43853, [2482ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.252:45547, [2482ms] Sub RPC ConnectToMaster: sending RPC to server master-127.6.85.253:35731, [2486ms] too many traces: truncated at 100 traces, deferred=Deferred@276826420(state=PENDING, result=null, callback=(continuation of Deferred@995042789 after retry RPC after error@927238483) -> (continuation of Deferred@1686896755 after retry RPC after error@1960148095) -> (continuation of Deferred@913776397 after retry RPC after error@627709569) -> (continuation of Deferred@560010695 after retry RPC after error@2122883856) -> (continuation of Deferred@1879655611 after retry RPC after error@86153765) -> (continuation of Deferred@751073387 after retry RPC after error@401592000) -> (continuation of Deferred@762932250 after retry RPC after error@1016801314) -> (continuation of Deferred@865274106 after retry RPC after error@1524849030) -> (continuation of Deferred@1008527228 after retry RPC after error@1495466725) -> (continuation of Deferred@1918637539 after retry RPC after error@850615623) -> (continuation of Deferred@1766812522 after retry RPC after error@683309196) -> (continuation of Deferred@987468661 after retry RPC after error@313174028) -> (continuation of Deferred@11306248 after retry RPC after error@2074731401) -> (continuation of Deferred@487561138 after retry RPC after error@1520608983) -> (continuation of Deferred@1044577358 after retry RPC after error@180614564) -> (continuation of Deferred@1804620105 after retry RPC after error@617338610) -> (continuation of Deferred@230878889 after retry RPC after error@406410336) -> (continuation of Deferred@1909081143 after retry RPC after error@189558970) -> (continuation of Deferred@299987164 after retry RPC after error@1126804692) -> (continuation of Deferred@1625400186 after retry RPC after error@146855548) -> (continuation of Deferred@305995816 after retry RPC after error@486306231) -> (continuation of Deferred@1911290657 after retry RPC after error@2140577878) -> (continuation of Deferred@80864236 after retry RPC after error@2069842696) -> (continuation of Deferred@1438877105 after retry RPC after error@522001231), errback=(continuation of Deferred@995042789 after retry RPC after error@927238483) -> (continuation of Deferred@1686896755 after retry RPC after error@1960148095) -> (continuation of Deferred@913776397 after retry RPC after error@627709569) -> (continuation of Deferred@560010695 after retry RPC after error@2122883856) -> (continuation of Deferred@1879655611 after retry RPC after error@86153765) -> (continuation of Deferred@751073387 after retry RPC after error@401592000) -> (continuation of Deferred@762932250 after retry RPC after error@1016801314) -> (continuation of Deferred@865274106 after retry RPC after error@1524849030) -> (continuation of Deferred@1008527228 after retry RPC after error@1495466725) -> (continuation of Deferred@1918637539 after retry RPC after error@850615623) -> (continuation of Deferred@1766812522 after retry RPC after error@683309196) -> (continuation of Deferred@987468661 after retry RPC after error@313174028) -> (continuation of Deferred@11306248 after retry RPC after error@2074731401) -> (continuation of Deferred@487561138 after retry RPC after error@1520608983) -> (continuation of Deferred@1044577358 after retry RPC after error@180614564) -> (continuation of Deferred@1804620105 after retry RPC after error@617338610) -> (continuation of Deferred@230878889 after retry RPC after error@406410336) -> (continuation of Deferred@1909081143 after retry RPC after error@189558970) -> (continuation of Deferred@299987164 after retry RPC after error@1126804692) -> (continuation of Deferred@1625400186 after retry RPC after error@146855548) -> (continuation of Deferred@305995816 after retry RPC after error@486306231) -> (continuation of Deferred@1911290657 after retry RPC after error@2140577878) -> (continuation of Deferred@80864236 after retry RPC after error@2069842696) -> (continuation of Deferred@1438877105 after retry RPC after error@522001231)))
at org.apache.kudu.client.KuduException.transformException(KuduException.java:110)
at org.apache.kudu.client.KuduClient.joinAndHandleException(KuduClient.java:564)
at org.apache.kudu.client.KuduClient.listTabletServers(KuduClient.java:263)
at org.apache.kudu.client.TestSecurity$3.get(TestSecurity.java:386)
at org.apache.kudu.test.junit.AssertHelpers.assertEventuallyTrue(AssertHelpers.java:40)
at org.apache.kudu.client.TestSecurity.assertEventualAuthenticationFailure(TestSecurity.java:378)
at org.apache.kudu.client.TestSecurity.testDoNotSwitchPrincipalsInExistingClient(TestSecurity.java:366)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Suppressed: org.apache.kudu.client.KuduException$OriginalException: Original asynchronous stack trace
at org.apache.kudu.client.AsyncKuduClient.tooManyAttemptsOrTimeout(AsyncKuduClient.java:1889)
at org.apache.kudu.client.AsyncKuduClient.delayedSendRpcToTablet(AsyncKuduClient.java:2337)
at org.apache.kudu.client.AsyncKuduClient.access$1500(AsyncKuduClient.java:299)
at org.apache.kudu.client.AsyncKuduClient$RetryRpcErrback.call(AsyncKuduClient.java:1520)
at org.apache.kudu.client.AsyncKuduClient$RetryRpcErrback.call(AsyncKuduClient.java:1510)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002)
at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:313)
at org.apache.kudu.client.ConnectToCluster.access$000(ConnectToCluster.java:51)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:387)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:376)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002)
at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:277)
at org.apache.kudu.client.KuduRpc.errback(KuduRpc.java:331)
at org.apache.kudu.client.RpcProxy.responseReceived(RpcProxy.java:256)
at org.apache.kudu.client.RpcProxy.access$000(RpcProxy.java:64)
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:158)
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:154)
at org.apache.kudu.client.Connection.cleanup(Connection.java:737)
at org.apache.kudu.client.Connection.exceptionCaught(Connection.java:455)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317)
at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:447)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.kudu.client.NoLeaderFoundException: Master config (127.6.85.254:43853,127.6.85.252:45547,127.6.85.253:35731) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:311)
at org.apache.kudu.client.ConnectToCluster.access$000(ConnectToCluster.java:51)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:387)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:376)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002)
at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:277)
at org.apache.kudu.client.KuduRpc.errback(KuduRpc.java:331)
at org.apache.kudu.client.RpcProxy.responseReceived(RpcProxy.java:256)
at org.apache.kudu.client.RpcProxy.access$000(RpcProxy.java:64)
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:158)
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:154)
at org.apache.kudu.client.Connection.cleanup(Connection.java:737)
at org.apache.kudu.client.Connection.exceptionCaught(Connection.java:455)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317)
at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:447)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
at org.apache.kudu.client.Negotiator.chooseAndInitializeSaslMech(Negotiator.java:565)
at org.apache.kudu.client.Negotiator.handleNegotiateResponse(Negotiator.java:433)
at org.apache.kudu.client.Negotiator.channelRead0(Negotiator.java:367)
at org.apache.kudu.client.Negotiator.channelRead0(Negotiator.java:94)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
... 30 more