07:43:02.983 [INFO - main] (KuduTestHarness.java:155) Creating a new MiniKuduCluster...
07:43:02.983 [INFO - main] (TempDirUtils.java:71) Using the temp directory defined by TEST_TMPDIR: /tmp/dist-test-taskNAFIDr/test-tmp
07:43:02.983 [INFO - main] (KuduBinaryLocator.java:83) Using Kudu binary directory specified by system property 'kuduBinDir': ../../../build/tsan/bin
07:43:02.983 [INFO - main] (MiniKuduCluster.java:217) Starting process: [/tmp/dist-test-taskNAFIDr/java/build/dist-test/../../../build/tsan/bin/kudu, test, mini_cluster, --serialization=pb]
07:43:02.985 [DEBUG - main] (MiniKuduCluster.java:185) Request: create_cluster {
num_masters: 3
num_tservers: 3
enable_kerberos: false
cluster_root: "/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356"
hms_mode: NONE
mini_kdc_options {
}
principal: "kudu"
mini_oidc_options {
}
}
07:43:03.305 [DEBUG - main] (MiniKuduCluster.java:195) Response:
07:43:03.305 [DEBUG - main] (MiniKuduCluster.java:185) Request: start_cluster {
}
07:43:03.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:03.308133 7663 mini_oidc.cc:152] Starting JWKS server
07:43:03.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:03.311910 7663 webserver.cc:460] Webserver started at http://127.0.0.1:34971/ using document root <none> and password file <none>
07:43:03.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:03.312408 7663 mini_oidc.cc:189] Starting OIDC Discovery server
07:43:03.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:03.314177 7663 webserver.cc:460] Webserver started at http://127.0.0.1:32923/ using document root <none> and password file <none>
07:43:03.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-06-20T07:43:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
07:43:03.355 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-06-20T07:43:03Z Disabled control of system clock
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:03.370157 7663 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/wal
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/data
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/logs
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/data/info.pb
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:43:03.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.7.123.254:37521
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.7.123.254
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.7.123.212:46711
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.7.123.254:37521,127.7.123.253:41641,127.7.123.252:38375
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:03.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:34971/jwks.json with env {}
07:43:03.668 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:03.668418 7675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:43:03.669 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:03.669021 7675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:43:03.669 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:03.669500 7675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:43:03.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:03.700987 7675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:43:03.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:03.701268 7675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:43:03.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:03.701469 7675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:43:03.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:03.701663 7675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:43:03.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:03.701850 7675 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:43:03.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:03.701992 7675 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:34971/jwks.json
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:03.737270 7675 master_runner.cc:386] Master server non-default flags:
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.7.123.212:46711
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/data
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/wal
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.7.123.254:37521,127.7.123.253:41641,127.7.123.252:38375
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.7.123.254:37521
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:43:03.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:34971/jwks.json
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/data/info.pb
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.7.123.254
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/logs
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 20 Jun 2026 06:17:27 UTC on ccycloud.quasar-mfmkhc.root.comops.site
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4456919
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:43:03.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:03.738557 7675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:43:03.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:03.740072 7675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:43:03.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:03.757135 7681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:04.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:03.757151 7682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:04.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:03.757485 7684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:04.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:03.758968 7675 server_base.cc:1034] running on GCE node
07:43:04.959 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.959319 7675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:43:04.962 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.962405 7675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:43:04.964 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.963828 7675 hybrid_clock.cc:648] HybridClock initialized: now 1781941384963795 us; error 51 us; skew 500 ppm
07:43:04.964 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.964691 7675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:43:04.974 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.974227 7675 webserver.cc:460] Webserver started at http://127.7.123.254:41571/ using document root <none> and password file <none>
07:43:04.975 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.975106 7675 fs_manager.cc:359] Metadata directory not provided
07:43:04.975 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.975291 7675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:43:04.975 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.975666 7675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:43:04.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.980482 7675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/data/instance:
07:43:04.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "142a5f40ee2c4bd89c3bbc46ea31b462"
07:43:04.980 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:04 on dist-test-slave-bjlg"
07:43:04.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.981516 7675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/wal/instance:
07:43:04.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "142a5f40ee2c4bd89c3bbc46ea31b462"
07:43:04.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:04 on dist-test-slave-bjlg"
07:43:04.989 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.989176 7675 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
07:43:04.995 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.995302 7691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:43:04.996 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.996596 7675 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.006s sys 0.000s
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.996949 7675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/data,/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/wal
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "142a5f40ee2c4bd89c3bbc46ea31b462"
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:04 on dist-test-slave-bjlg"
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:04.997279 7675 fs_report.cc:383] FS layout report
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/wal
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/wal
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/data/data
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:43:04.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:43:04.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:43:04.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:43:04.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:43:04.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:43:05.032 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.032120 7675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:43:05.034 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.033802 7675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:43:05.034 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.034282 7675 kserver.cc:163] Server-wide thread pool size limit: 3276
07:43:05.105 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.105412 7675 rpc_server.cc:307] RPC server started. Bound to: 127.7.123.254:37521
07:43:05.105 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.105512 7742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.123.254:37521 every 8 connection(s)
07:43:05.108 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.108070 7675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/data/info.pb
07:43:05.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.113291 7743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
07:43:05.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.115819 7663 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu as pid 7675
07:43:05.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.116300 7663 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-0/wal/instance
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.119844 7663 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/wal
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/data
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/logs
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/data/info.pb
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.7.123.253:41641
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.7.123.253
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:05.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.7.123.212:46711
07:43:05.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:05.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:05.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:05.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:05.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.7.123.254:37521,127.7.123.253:41641,127.7.123.252:38375
07:43:05.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:05.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:34971/jwks.json with env {}
07:43:05.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.124336 7743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } has no permanent_uuid. Determining permanent_uuid...
07:43:05.154 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.153851 7743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } has no permanent_uuid. Determining permanent_uuid...
07:43:05.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.156311 7694 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.123.253:41641: connect: Connection refused (error 111)
07:43:05.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.160456 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.253:41641: Network error: Client connection negotiation failed: client connection to 127.7.123.253:41641: connect: Connection refused (error 111)
07:43:05.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.210026 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } attempt: 1
07:43:05.214 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.213966 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.253:41641: Network error: Client connection negotiation failed: client connection to 127.7.123.253:41641: connect: Connection refused (error 111)
07:43:05.283 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.282447 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } attempt: 2
07:43:05.286 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.286234 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.253:41641: Network error: Client connection negotiation failed: client connection to 127.7.123.253:41641: connect: Connection refused (error 111)
07:43:05.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.377699 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } attempt: 3
07:43:05.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.381539 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.253:41641: Network error: Client connection negotiation failed: client connection to 127.7.123.253:41641: connect: Connection refused (error 111)
07:43:05.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.429072 7745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:43:05.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.429721 7745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:43:05.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.430203 7745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:43:05.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.460850 7745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:43:05.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.461118 7745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:43:05.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.461311 7745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:43:05.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.461508 7745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:43:05.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.461684 7745 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:43:05.461 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.461825 7745 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:34971/jwks.json
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.496745 7745 master_runner.cc:386] Master server non-default flags:
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.7.123.212:46711
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/data
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/wal
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.7.123.254:37521,127.7.123.253:41641,127.7.123.252:38375
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.7.123.253:41641
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:34971/jwks.json
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/data/info.pb
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.7.123.253
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/logs
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:43:05.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:43:05.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:43:05.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:43:05.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:43:05.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 20 Jun 2026 06:17:27 UTC on ccycloud.quasar-mfmkhc.root.comops.site
07:43:05.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4456919
07:43:05.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:43:05.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.497953 7745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:43:05.499 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.499465 7745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:43:05.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.516911 7754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:05.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.524955 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } attempt: 4
07:43:05.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.528875 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.253:41641: Network error: Client connection negotiation failed: client connection to 127.7.123.253:41641: connect: Connection refused (error 111)
07:43:05.828 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.828291 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } attempt: 5
07:43:05.832 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.832175 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.253:41641: Network error: Client connection negotiation failed: client connection to 127.7.123.253:41641: connect: Connection refused (error 111)
07:43:06.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.379859 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } attempt: 6
07:43:06.387 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:06.386883 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.253:41641: Network error: Client connection negotiation failed: client connection to 127.7.123.253:41641: connect: Connection refused (error 111)
07:43:06.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:05.518613 7745 server_base.cc:1034] running on GCE node
07:43:06.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.516960 7757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:06.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:05.517038 7755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:06.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.718809 7745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:43:06.722 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.722146 7745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:43:06.723 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.723631 7745 hybrid_clock.cc:648] HybridClock initialized: now 1781941386723598 us; error 50 us; skew 500 ppm
07:43:06.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.724486 7745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:43:06.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.731316 7745 webserver.cc:460] Webserver started at http://127.7.123.253:34721/ using document root <none> and password file <none>
07:43:06.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.732192 7745 fs_manager.cc:359] Metadata directory not provided
07:43:06.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.732375 7745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:43:06.733 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.732839 7745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:43:06.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.737798 7745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/data/instance:
07:43:06.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8876f7472b8240caaf25572b79391b94"
07:43:06.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:06 on dist-test-slave-bjlg"
07:43:06.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.738862 7745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/wal/instance:
07:43:06.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8876f7472b8240caaf25572b79391b94"
07:43:06.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:06 on dist-test-slave-bjlg"
07:43:06.746 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.746179 7745 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
07:43:06.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.752197 7766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:43:06.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.753537 7745 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
07:43:06.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.753995 7745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/data,/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/wal
07:43:06.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "8876f7472b8240caaf25572b79391b94"
07:43:06.754 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:06 on dist-test-slave-bjlg"
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.754415 7745 fs_report.cc:383] FS layout report
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/wal
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/wal
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/data/data
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:43:06.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:43:06.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:43:06.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:43:06.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.798138 7745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:43:06.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.799687 7745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:43:06.800 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.800192 7745 kserver.cc:163] Server-wide thread pool size limit: 3276
07:43:06.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.869870 7745 rpc_server.cc:307] RPC server started. Bound to: 127.7.123.253:41641
07:43:06.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.870072 7817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.123.253:41641 every 8 connection(s)
07:43:06.873 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.872953 7745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/data/info.pb
07:43:06.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.876827 7663 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu as pid 7745
07:43:06.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.877326 7663 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-1/wal/instance
07:43:06.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.879628 7818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
07:43:06.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.880406 7663 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:06.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:06.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/wal
07:43:06.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/data
07:43:06.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:43:06.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:43:06.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:43:06.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:06.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:06.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:06.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/logs
07:43:06.880 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/data/info.pb
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.7.123.252:38375
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.7.123.252
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.7.123.212:46711
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.7.123.254:37521,127.7.123.253:41641,127.7.123.252:38375
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:06.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:34971/jwks.json with env {}
07:43:06.895 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.893771 7818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } has no permanent_uuid. Determining permanent_uuid...
07:43:06.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.918124 7818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } has no permanent_uuid. Determining permanent_uuid...
07:43:06.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.934667 7818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } has no permanent_uuid. Determining permanent_uuid...
07:43:06.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:06.937163 7767 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:06.941 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:06.941069 7818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:06.991 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:06.990708 7818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 1
07:43:06.995 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:06.995187 7818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:07.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.063711 7818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 2
07:43:07.067 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.067488 7818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:07.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.158902 7818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 3
07:43:07.162 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.162631 7818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:07.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.210019 7820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:43:07.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.210563 7820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:43:07.211 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.211026 7820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:43:07.241 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.241636 7820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:43:07.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.241914 7820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:43:07.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.242112 7820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:43:07.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.242377 7820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:43:07.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.242587 7820 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:43:07.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.242728 7820 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:34971/jwks.json
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.277917 7820 master_runner.cc:386] Master server non-default flags:
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.7.123.212:46711
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/data
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/wal
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.7.123.254:37521,127.7.123.253:41641,127.7.123.252:38375
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.7.123.252:38375
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:34971/jwks.json
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/data/info.pb
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.7.123.252
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/logs
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:43:07.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:43:07.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:43:07.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:43:07.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 20 Jun 2026 06:17:27 UTC on ccycloud.quasar-mfmkhc.root.comops.site
07:43:07.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4456919
07:43:07.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:43:07.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.279098 7820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:43:07.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.280676 7820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:43:07.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.298066 7831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:07.306 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.306056 7818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 4
07:43:07.310 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.310027 7818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:07.448 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.447544 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } attempt: 7
07:43:07.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.465226 7743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } has no permanent_uuid. Determining permanent_uuid...
07:43:07.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.471913 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:07.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.530357 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 1
07:43:07.535 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.534782 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:07.610 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.609469 7818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 5
07:43:07.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.613956 7818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:07.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.616194 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 2
07:43:07.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.620613 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:07.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.706013 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 3
07:43:07.711 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.710723 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:07.851 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.851188 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 4
07:43:07.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.855998 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:08.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.139446 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 5
07:43:08.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:08.144220 7743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:08.161 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.161335 7818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 6
07:43:08.168 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:08.167876 7818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.123.252:38375: Network error: Client connection negotiation failed: client connection to 127.7.123.252:38375: connect: Connection refused (error 111)
07:43:08.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.298058 7830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:08.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:07.299952 7820 server_base.cc:1034] running on GCE node
07:43:08.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:07.298180 7833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:08.523 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.522814 7820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:43:08.525 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.525606 7820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:43:08.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.527006 7820 hybrid_clock.cc:648] HybridClock initialized: now 1781941388526964 us; error 51 us; skew 500 ppm
07:43:08.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.527882 7820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:43:08.534 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.534236 7820 webserver.cc:460] Webserver started at http://127.7.123.252:44113/ using document root <none> and password file <none>
07:43:08.535 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.535168 7820 fs_manager.cc:359] Metadata directory not provided
07:43:08.535 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.535403 7820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:43:08.536 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.535879 7820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:43:08.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.540906 7820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/data/instance:
07:43:08.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "548c59e337d9421b998d3ebacc1daae1"
07:43:08.541 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:08 on dist-test-slave-bjlg"
07:43:08.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.542033 7820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/wal/instance:
07:43:08.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "548c59e337d9421b998d3ebacc1daae1"
07:43:08.542 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:08 on dist-test-slave-bjlg"
07:43:08.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.549372 7820 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
07:43:08.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.555112 7845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:43:08.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.556277 7820 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.556703 7820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/data,/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/wal
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "548c59e337d9421b998d3ebacc1daae1"
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:08 on dist-test-slave-bjlg"
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.557072 7820 fs_report.cc:383] FS layout report
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/wal
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/wal
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/data/data
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:43:08.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:43:08.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:43:08.594 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.593940 7820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:43:08.595 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.595504 7820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:43:08.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.596007 7820 kserver.cc:163] Server-wide thread pool size limit: 3276
07:43:08.666 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.666672 7820 rpc_server.cc:307] RPC server started. Bound to: 127.7.123.252:38375
07:43:08.667 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.666800 7896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.123.252:38375 every 8 connection(s)
07:43:08.669 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.669432 7820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/data/info.pb
07:43:08.674 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.674636 7897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
07:43:08.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.677088 7663 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu as pid 7820
07:43:08.677 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.677588 7663 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/master-2/wal/instance
07:43:08.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.687695 7897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } has no permanent_uuid. Determining permanent_uuid...
07:43:08.697 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.696825 7743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 6
07:43:08.721 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.721073 7897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } has no permanent_uuid. Determining permanent_uuid...
07:43:08.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.736815 7897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } has no permanent_uuid. Determining permanent_uuid...
07:43:08.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.742228 7743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462: Bootstrap starting.
07:43:08.748 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.748188 7743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462: Neither blocks nor log segments found. Creating new log.
07:43:08.751 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.750797 7743 log.cc:824] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462: Log is configured to *not* fsync() on all Append() calls
07:43:08.755 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.755642 7743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462: No bootstrap required, opened a new log
07:43:08.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.757926 7897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1: Bootstrap starting.
07:43:08.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.764508 7897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1: Neither blocks nor log segments found. Creating new log.
07:43:08.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.766646 7897 log.cc:824] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1: Log is configured to *not* fsync() on all Append() calls
07:43:08.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.771791 7897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1: No bootstrap required, opened a new log
07:43:08.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.780459 7743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } }
07:43:08.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.781239 7743 raft_consensus.cc:374] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
07:43:08.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.781517 7743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142a5f40ee2c4bd89c3bbc46ea31b462, State: Initialized, Role: FOLLOWER
07:43:08.782 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.782245 7743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } }
07:43:08.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.785228 7909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } } }
07:43:08.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.786067 7909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [sys.catalog]: This master's current role is: FOLLOWER
07:43:08.787 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.786916 7743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [sys.catalog]: configured and running, proceeding with master startup.
07:43:08.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:08.793179 7697 tablet.cc:2369] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462: Can't schedule compaction. Clean time has not been advanced past its initial value.
07:43:08.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.795918 7897 raft_consensus.cc:348] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } }
07:43:08.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.796622 7897 raft_consensus.cc:374] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
07:43:08.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.796974 7897 raft_consensus.cc:729] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 548c59e337d9421b998d3ebacc1daae1, State: Initialized, Role: FOLLOWER
07:43:08.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.797923 7897 consensus_queue.cc:260] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } }
07:43:08.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.800945 7915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } } }
07:43:08.802 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.801780 7915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [sys.catalog]: This master's current role is: FOLLOWER
07:43:08.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.802876 7897 sys_catalog.cc:564] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [sys.catalog]: configured and running, proceeding with master startup.
07:43:08.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:08.823138 7923 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
07:43:08.823 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:08.823693 7923 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
07:43:08.849 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.817754 7675 master_runner.cc:186] Error getting master registration for 127.7.123.253:41641: OK, instance_id { permanent_uuid: "8876f7472b8240caaf25572b79391b94" instance_seqno: 1781941386850523 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
07:43:08.849 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.849339 7675 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
07:43:08.851 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:08.851199 7851 tablet.cc:2369] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1: Can't schedule compaction. Clean time has not been advanced past its initial value.
07:43:08.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:08.855679 7933 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
07:43:08.856 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:08.856005 7933 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
07:43:08.885 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.849067 7820 master_runner.cc:186] Error getting master registration for 127.7.123.253:41641: OK, instance_id { permanent_uuid: "8876f7472b8240caaf25572b79391b94" instance_seqno: 1781941386850523 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
07:43:08.885 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.885222 7820 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
07:43:08.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.989500 7909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
07:43:08.993 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.990104 7909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } }
07:43:08.996 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.994793 7872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "548c59e337d9421b998d3ebacc1daae1" is_pre_election: true
07:43:08.996 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.995777 7872 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 142a5f40ee2c4bd89c3bbc46ea31b462 in term 0.
07:43:08.997 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.996793 7909 leader_election.cc:288] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8876f7472b8240caaf25572b79391b94 (127.7.123.253:41641), 548c59e337d9421b998d3ebacc1daae1 (127.7.123.252:38375)
07:43:08.999 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.998581 7692 leader_election.cc:302] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142a5f40ee2c4bd89c3bbc46ea31b462, 548c59e337d9421b998d3ebacc1daae1; no voters:
07:43:09.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:08.999646 7909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [term 0 FOLLOWER]: Leader pre-election won for term 1
07:43:09.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.000005 7909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
07:43:09.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.000324 7909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [term 0 FOLLOWER]: Advancing to term 1
07:43:09.007 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.006886 7909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } }
07:43:09.009 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.008930 7909 leader_election.cc:288] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [CANDIDATE]: Term 1 election: Requested vote from peers 8876f7472b8240caaf25572b79391b94 (127.7.123.253:41641), 548c59e337d9421b998d3ebacc1daae1 (127.7.123.252:38375)
07:43:09.010 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.010515 7872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "548c59e337d9421b998d3ebacc1daae1"
07:43:09.011 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.011068 7872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [term 0 FOLLOWER]: Advancing to term 1
07:43:09.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.023049 7872 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 142a5f40ee2c4bd89c3bbc46ea31b462 in term 1.
07:43:09.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.024268 7692 leader_election.cc:302] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 142a5f40ee2c4bd89c3bbc46ea31b462, 548c59e337d9421b998d3ebacc1daae1; no voters:
07:43:09.026 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.025887 7909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [term 1 FOLLOWER]: Leader election won for term 1
07:43:09.031 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.031507 7909 raft_consensus.cc:686] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [term 1 LEADER]: Becoming Leader. State: Replica: 142a5f40ee2c4bd89c3bbc46ea31b462, State: Running, Role: LEADER
07:43:09.033 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.032688 7909 consensus_queue.cc:237] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } }
07:43:09.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.044584 7935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 142a5f40ee2c4bd89c3bbc46ea31b462. Latest consensus state: current_term: 1 leader_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } } }
07:43:09.045 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.045439 7935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [sys.catalog]: This master's current role is: LEADER
07:43:09.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.049392 7937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
07:43:09.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.061053 7937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
07:43:09.087 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.086645 7872 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [term 1 FOLLOWER]: Refusing update from remote peer 142a5f40ee2c4bd89c3bbc46ea31b462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
07:43:09.091 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.090042 7909 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
07:43:09.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.125584 7915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 142a5f40ee2c4bd89c3bbc46ea31b462. Latest consensus state: current_term: 1 leader_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } } }
07:43:09.128 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.126322 7915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [sys.catalog]: This master's current role is: FOLLOWER
07:43:09.134 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.133271 7938 mvcc.cc:205] Tried to move back new op lower bound from 7298831929668444160 to 7298831929487765504. Current Snapshot: MvccSnapshot[applied={T|T < 7298831929668444160}]
07:43:09.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.137339 7935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } } }
07:43:09.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.138082 7935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [sys.catalog]: This master's current role is: LEADER
07:43:09.139 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.138861 7939 mvcc.cc:205] Tried to move back new op lower bound from 7298831929668444160 to 7298831929487765504. Current Snapshot: MvccSnapshot[applied={T|T < 7298831929668444160}]
07:43:09.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.140111 7937 catalog_manager.cc:1347] Generated new cluster ID: 2bdfa44de5c0443cb123c69f68121397
07:43:09.140 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.140444 7937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
07:43:09.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.149365 7915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } } }
07:43:09.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.150213 7915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1 [sys.catalog]: This master's current role is: FOLLOWER
07:43:09.229 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.228468 7818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } attempt: 7
07:43:09.237 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.237347 7937 catalog_manager.cc:1370] Generated new certificate authority record
07:43:09.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.240175 7937 catalog_manager.cc:1504] Loading token signing keys...
07:43:09.282 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.281824 7818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94: Bootstrap starting.
07:43:09.289 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.288961 7818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94: Neither blocks nor log segments found. Creating new log.
07:43:09.289 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.289465 7937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462: Generated new TSK 0
07:43:09.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.290848 7937 catalog_manager.cc:1514] Initializing in-progress tserver states...
07:43:09.298 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.298096 7818 log.cc:824] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94: Log is configured to *not* fsync() on all Append() calls
07:43:09.309 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.309592 7818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94: No bootstrap required, opened a new log
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.339123 7663 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/wal
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/data
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/logs
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/data/info.pb
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:43:09.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.7.123.193:0
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.7.123.193
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.7.123.193
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.7.123.254:37521,127.7.123.253:41641,127.7.123.252:38375
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.7.123.212:46711
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:09.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
07:43:09.347 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.347028 7818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } }
07:43:09.348 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.348047 7818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
07:43:09.348 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.348434 7818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8876f7472b8240caaf25572b79391b94, State: Initialized, Role: FOLLOWER
07:43:09.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.349582 7818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } }
07:43:09.365 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.365535 7818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [sys.catalog]: configured and running, proceeding with master startup.
07:43:09.368 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.366202 7791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [term 0 FOLLOWER]: Advancing to term 1
07:43:09.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.369004 7792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8876f7472b8240caaf25572b79391b94"
07:43:09.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.368103 7793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8876f7472b8240caaf25572b79391b94" is_pre_election: true
07:43:09.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.380748 7791 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [term 1 FOLLOWER]: Refusing update from remote peer 142a5f40ee2c4bd89c3bbc46ea31b462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
07:43:09.384 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.382632 7909 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
07:43:09.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.486785 7949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 142a5f40ee2c4bd89c3bbc46ea31b462. Latest consensus state: current_term: 1 leader_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } } }
07:43:09.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.487761 7949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [sys.catalog]: This master's current role is: FOLLOWER
07:43:09.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.489111 7954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } } }
07:43:09.490 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.489907 7954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [sys.catalog]: This master's current role is: FOLLOWER
07:43:09.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.490860 7964 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
07:43:09.492 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.490813 7945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } } }
07:43:09.493 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.492959 7945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94 [sys.catalog]: This master's current role is: FOLLOWER
07:43:09.776 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.774230 7909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "142a5f40ee2c4bd89c3bbc46ea31b462" member_type: VOTER last_known_addr { host: "127.7.123.254" port: 37521 } } peers { permanent_uuid: "8876f7472b8240caaf25572b79391b94" member_type: VOTER last_known_addr { host: "127.7.123.253" port: 41641 } } peers { permanent_uuid: "548c59e337d9421b998d3ebacc1daae1" member_type: VOTER last_known_addr { host: "127.7.123.252" port: 38375 } } }
07:43:09.778 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.775043 7909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 142a5f40ee2c4bd89c3bbc46ea31b462 [sys.catalog]: This master's current role is: LEADER
07:43:09.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.861595 7933 catalog_manager.cc:1259] Loaded cluster ID: 2bdfa44de5c0443cb123c69f68121397
07:43:09.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.862087 7933 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1: loading cluster ID for follower catalog manager: success
07:43:09.870 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.870133 7933 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1: acquiring CA information for follower catalog manager: success
07:43:09.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:09.876041 7933 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 548c59e337d9421b998d3ebacc1daae1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:43:09.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:09.937964 7946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:43:09.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:09.938630 7946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:43:09.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:09.939399 7946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:43:09.999 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:09.998714 7946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:43:10.000 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:10.000138 7946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.123.193
07:43:10.061 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:10.061285 7946 tablet_server_runner.cc:78] Tablet server non-default flags:
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.7.123.212:46711
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/data
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/wal
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.7.123.193:0
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/data/info.pb
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.7.123.193
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.7.123.254:37521,127.7.123.253:41641,127.7.123.252:38375
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.7.123.193
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/logs
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
07:43:10.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:43:10.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:43:10.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:43:10.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 20 Jun 2026 06:17:27 UTC on ccycloud.quasar-mfmkhc.root.comops.site
07:43:10.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4456919
07:43:10.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:43:10.063 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:10.062868 7946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:43:10.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:10.064941 7946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:43:10.089 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:10.088693 7970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:10.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:10.508220 7964 catalog_manager.cc:1259] Loaded cluster ID: 2bdfa44de5c0443cb123c69f68121397
07:43:10.512 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:10.510149 7964 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94: loading cluster ID for follower catalog manager: success
07:43:10.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:10.527673 7964 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94: acquiring CA information for follower catalog manager: success
07:43:10.545 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:10.545281 7964 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8876f7472b8240caaf25572b79391b94: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:43:12.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:11.483862 7969 debug-util.cc:398] Leaking SignalData structure 0x7b0800014860 after lost signal to thread 7946
07:43:12.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:12.239965 7946 thread.cc:636] OpenStack (cloud detector) Time spent creating pthread: real 2.149s user 0.594s sys 1.053s
07:43:12.240 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:12.240329 7946 thread.cc:602] OpenStack (cloud detector) Time spent starting thread: real 2.149s user 0.594s sys 1.053s
07:43:12.242 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:12.242396 7973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:12.248 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:12.248534 7972 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
07:43:12.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.248662 7946 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
07:43:12.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:10.091511 7971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:12.251 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.250769 7946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:43:12.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.259965 7946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:43:12.261 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.261524 7946 hybrid_clock.cc:648] HybridClock initialized: now 1781941392261501 us; error 65 us; skew 500 ppm
07:43:12.262 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.262665 7946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:43:12.282 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.281879 7946 webserver.cc:460] Webserver started at http://127.7.123.193:43733/ using document root <none> and password file <none>
07:43:12.283 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.283329 7946 fs_manager.cc:359] Metadata directory not provided
07:43:12.284 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.284143 7946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:43:12.285 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.284861 7946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:43:12.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.291949 7946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/data/instance:
07:43:12.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3b1779592d624410a3ad7f3a4168a53a"
07:43:12.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:12 on dist-test-slave-bjlg"
07:43:12.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.293581 7946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/wal/instance:
07:43:12.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3b1779592d624410a3ad7f3a4168a53a"
07:43:12.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:12 on dist-test-slave-bjlg"
07:43:12.309 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.308713 7946 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.001s sys 0.013s
07:43:12.322 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.322052 7981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:43:12.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.323609 7946 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.006s
07:43:12.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.324110 7946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/data,/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/wal
07:43:12.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "3b1779592d624410a3ad7f3a4168a53a"
07:43:12.324 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:12 on dist-test-slave-bjlg"
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.324733 7946 fs_report.cc:383] FS layout report
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/wal
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/wal
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/data/data
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:43:12.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:43:12.420 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.420467 7946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:43:12.423 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.422904 7946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:43:12.423 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.423640 7946 kserver.cc:163] Server-wide thread pool size limit: 3276
07:43:12.433 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.432874 7946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
07:43:12.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.444671 7946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
07:43:12.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.445158 7946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
07:43:12.445 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.445650 7946 ts_tablet_manager.cc:610] Registered 0 tablets
07:43:12.446 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.445964 7946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
07:43:12.869 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.869156 7946 rpc_server.cc:307] RPC server started. Bound to: 127.7.123.193:41763
07:43:12.872 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.869305 8093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.123.193:41763 every 8 connection(s)
07:43:12.873 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.873309 7946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/data/info.pb
07:43:12.882 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.882520 7663 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu as pid 7946
07:43:12.883 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.882969 7663 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-0/wal/instance
07:43:12.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.893187 7663 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:12.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:12.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/wal
07:43:12.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/data
07:43:12.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:43:12.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:43:12.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/logs
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/data/info.pb
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.7.123.194:0
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.7.123.194
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.7.123.194
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:12.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.7.123.254:37521,127.7.123.253:41641,127.7.123.252:38375
07:43:12.895 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.7.123.212:46711
07:43:12.895 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:12.895 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:12.895 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
07:43:12.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.965335 8095 heartbeater.cc:347] Connected to a master server at 127.7.123.253:41641
07:43:12.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.965847 8095 heartbeater.cc:464] Registering TS with master...
07:43:12.968 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.967038 8095 heartbeater.cc:511] Master 127.7.123.253:41641 requested a full tablet report, sending...
07:43:12.971 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.970530 7783 ts_manager.cc:194] Registered new tserver with Master: 3b1779592d624410a3ad7f3a4168a53a (127.7.123.193:41763)
07:43:12.979 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.977152 8099 heartbeater.cc:347] Connected to a master server at 127.7.123.254:37521
07:43:12.979 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.977487 8099 heartbeater.cc:464] Registering TS with master...
07:43:12.979 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.978322 8099 heartbeater.cc:511] Master 127.7.123.254:37521 requested a full tablet report, sending...
07:43:12.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.980695 7708 ts_manager.cc:194] Registered new tserver with Master: 3b1779592d624410a3ad7f3a4168a53a (127.7.123.193:41763)
07:43:12.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.981405 8094 heartbeater.cc:347] Connected to a master server at 127.7.123.252:38375
07:43:12.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.981724 8094 heartbeater.cc:464] Registering TS with master...
07:43:12.986 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.982460 8094 heartbeater.cc:511] Master 127.7.123.252:38375 requested a full tablet report, sending...
07:43:12.986 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.983407 7708 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.123.193:59443
07:43:12.986 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:12.985427 7862 ts_manager.cc:194] Registered new tserver with Master: 3b1779592d624410a3ad7f3a4168a53a (127.7.123.193:41763)
07:43:13.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:13.296463 8098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:43:13.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:13.297046 8098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:43:13.297 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:13.297649 8098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:43:13.333 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:13.333546 8098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:43:13.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:13.334349 8098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.123.194
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:13.373783 8098 tablet_server_runner.cc:78] Tablet server non-default flags:
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.7.123.212:46711
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/data
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/wal
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.7.123.194:0
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/data/info.pb
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.7.123.194
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.7.123.254:37521,127.7.123.253:41641,127.7.123.252:38375
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:13.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:43:13.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:43:13.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:13.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.7.123.194
07:43:13.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/logs
07:43:13.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:13.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:43:13.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:43:13.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
07:43:13.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:43:13.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:43:13.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:43:13.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 20 Jun 2026 06:17:27 UTC on ccycloud.quasar-mfmkhc.root.comops.site
07:43:13.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4456919
07:43:13.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:43:13.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:13.375053 8098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:43:13.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:13.377044 8098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:43:13.398 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:13.398476 8114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:13.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:13.990384 8099 heartbeater.cc:503] Master 127.7.123.254:37521 was elected leader, sending a full tablet report...
07:43:14.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:13.398917 8111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:14.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:13.399377 8112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:14.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:14.620476 8113 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
07:43:14.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.620497 8098 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
07:43:14.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.624354 8098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:43:14.626 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.626606 8098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:43:14.628 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.627966 8098 hybrid_clock.cc:648] HybridClock initialized: now 1781941394627934 us; error 51 us; skew 500 ppm
07:43:14.628 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.628820 8098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:43:14.635 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.635047 8098 webserver.cc:460] Webserver started at http://127.7.123.194:45151/ using document root <none> and password file <none>
07:43:14.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.635973 8098 fs_manager.cc:359] Metadata directory not provided
07:43:14.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.636165 8098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:43:14.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.636605 8098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:43:14.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.641444 8098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/data/instance:
07:43:14.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "bd22f8702ddd4926a93ec2835ce0f94d"
07:43:14.641 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:14 on dist-test-slave-bjlg"
07:43:14.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.642647 8098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/wal/instance:
07:43:14.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "bd22f8702ddd4926a93ec2835ce0f94d"
07:43:14.642 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:14 on dist-test-slave-bjlg"
07:43:14.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.649748 8098 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
07:43:14.655 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.655650 8122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:43:14.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.656919 8098 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
07:43:14.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.657204 8098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/data,/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/wal
07:43:14.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "bd22f8702ddd4926a93ec2835ce0f94d"
07:43:14.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:14 on dist-test-slave-bjlg"
07:43:14.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.657508 8098 fs_report.cc:383] FS layout report
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/wal
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/wal
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/data/data
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:43:14.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:43:14.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.706334 8098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:43:14.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.707718 8098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:43:14.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.708194 8098 kserver.cc:163] Server-wide thread pool size limit: 3276
07:43:14.710 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.710640 8098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
07:43:14.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.714813 8098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
07:43:14.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.715039 8098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
07:43:14.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.715267 8098 ts_tablet_manager.cc:610] Registered 0 tablets
07:43:14.715 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.715420 8098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
07:43:14.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.857537 8098 rpc_server.cc:307] RPC server started. Bound to: 127.7.123.194:43897
07:43:14.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.857641 8234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.123.194:43897 every 8 connection(s)
07:43:14.860 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.860255 8098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/data/info.pb
07:43:14.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.865700 7663 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu as pid 8098
07:43:14.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.866361 7663 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-1/wal/instance
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.889074 7663 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/wal
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/data
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/logs
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/data/info.pb
07:43:14.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) tserver
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.7.123.195:0
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.7.123.195
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.7.123.195
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.7.123.254:37521,127.7.123.253:41641,127.7.123.252:38375
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.7.123.212:46711
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:14.890 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin with env {}
07:43:14.919 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.919400 8235 heartbeater.cc:347] Connected to a master server at 127.7.123.252:38375
07:43:14.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.919919 8235 heartbeater.cc:464] Registering TS with master...
07:43:14.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.921253 8235 heartbeater.cc:511] Master 127.7.123.252:38375 requested a full tablet report, sending...
07:43:14.925 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.924428 7862 ts_manager.cc:194] Registered new tserver with Master: bd22f8702ddd4926a93ec2835ce0f94d (127.7.123.194:43897)
07:43:14.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.927783 8236 heartbeater.cc:347] Connected to a master server at 127.7.123.253:41641
07:43:14.928 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.928140 8236 heartbeater.cc:464] Registering TS with master...
07:43:14.929 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.929055 8236 heartbeater.cc:511] Master 127.7.123.253:41641 requested a full tablet report, sending...
07:43:14.932 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.931731 7783 ts_manager.cc:194] Registered new tserver with Master: bd22f8702ddd4926a93ec2835ce0f94d (127.7.123.194:43897)
07:43:14.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.932719 8237 heartbeater.cc:347] Connected to a master server at 127.7.123.254:37521
07:43:14.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.933068 8237 heartbeater.cc:464] Registering TS with master...
07:43:14.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.933769 8237 heartbeater.cc:511] Master 127.7.123.254:37521 requested a full tablet report, sending...
07:43:14.935 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.935662 7708 ts_manager.cc:194] Registered new tserver with Master: bd22f8702ddd4926a93ec2835ce0f94d (127.7.123.194:43897)
07:43:14.937 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:14.937256 7708 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.123.194:59253
07:43:15.243 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:15.242674 8245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:43:15.243 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:15.243292 8245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:43:15.243 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:15.243732 8245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:43:15.275 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:15.275686 8245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:43:15.276 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:15.276721 8245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.123.195
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:15.312093 8245 tablet_server_runner.cc:78] Tablet server non-default flags:
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.7.123.212:46711
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/data
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/wal
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.7.123.195:0
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/data/info.pb
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.7.123.195
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tserver_master_addrs=127.7.123.254:37521,127.7.123.253:41641,127.7.123.252:38375
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --local_ip_for_outbound_sockets=127.7.123.195
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/logs
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:15.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:43:15.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:43:15.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Tablet server version:
07:43:15.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:43:15.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:43:15.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:43:15.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 20 Jun 2026 06:17:27 UTC on ccycloud.quasar-mfmkhc.root.comops.site
07:43:15.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4456919
07:43:15.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:43:15.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:15.313349 8245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:43:15.315 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:15.315059 8245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:43:15.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:15.333756 8252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:15.940 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:15.940536 8237 heartbeater.cc:503] Master 127.7.123.254:37521 was elected leader, sending a full tablet report...
07:43:16.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:15.333746 8254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:16.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:15.335233 8251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:16.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:15.335574 8245 server_base.cc:1034] running on GCE node
07:43:16.543 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.542928 8245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:43:16.545 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.545142 8245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:43:16.546 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.546521 8245 hybrid_clock.cc:648] HybridClock initialized: now 1781941396546472 us; error 67 us; skew 500 ppm
07:43:16.547 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.547322 8245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:43:16.553 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.553596 8245 webserver.cc:460] Webserver started at http://127.7.123.195:34727/ using document root <none> and password file <none>
07:43:16.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.554426 8245 fs_manager.cc:359] Metadata directory not provided
07:43:16.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.554615 8245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:43:16.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.555001 8245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:43:16.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.559919 8245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/data/instance:
07:43:16.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "e774593d949d4b81b9bdb3e30ad22f15"
07:43:16.560 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:16 on dist-test-slave-bjlg"
07:43:16.561 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.561059 8245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/wal/instance:
07:43:16.561 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "e774593d949d4b81b9bdb3e30ad22f15"
07:43:16.561 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:16 on dist-test-slave-bjlg"
07:43:16.569 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.568805 8245 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
07:43:16.574 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.574484 8262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:43:16.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.575786 8245 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
07:43:16.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.576128 8245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/data,/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/wal
07:43:16.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "e774593d949d4b81b9bdb3e30ad22f15"
07:43:16.576 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:16 on dist-test-slave-bjlg"
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.576438 8245 fs_report.cc:383] FS layout report
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/wal
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/wal
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/data/data
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:43:16.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:43:16.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.613008 8245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:43:16.614 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.614356 8245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:43:16.615 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.614900 8245 kserver.cc:163] Server-wide thread pool size limit: 3276
07:43:16.617 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.617622 8245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
07:43:16.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.621865 8245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
07:43:16.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.622097 8245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
07:43:16.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.622416 8245 ts_tablet_manager.cc:610] Registered 0 tablets
07:43:16.622 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.622610 8245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
07:43:16.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.770454 8245 rpc_server.cc:307] RPC server started. Bound to: 127.7.123.195:36859
07:43:16.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.770591 8374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.123.195:36859 every 8 connection(s)
07:43:16.773 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.773536 8245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/data/info.pb
07:43:16.780 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.780294 7663 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu as pid 8245
07:43:16.781 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.780870 7663 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster290579574679204356/ts-2/wal/instance
07:43:16.828 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.828097 8376 heartbeater.cc:347] Connected to a master server at 127.7.123.253:41641
07:43:16.828 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.828658 8376 heartbeater.cc:464] Registering TS with master...
07:43:16.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.828802 8377 heartbeater.cc:347] Connected to a master server at 127.7.123.254:37521
07:43:16.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.829175 8377 heartbeater.cc:464] Registering TS with master...
07:43:16.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.829630 8376 heartbeater.cc:511] Master 127.7.123.253:41641 requested a full tablet report, sending...
07:43:16.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.830050 8377 heartbeater.cc:511] Master 127.7.123.254:37521 requested a full tablet report, sending...
07:43:16.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.830899 8375 heartbeater.cc:347] Connected to a master server at 127.7.123.252:38375
07:43:16.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.831229 8375 heartbeater.cc:464] Registering TS with master...
07:43:16.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.832044 7708 ts_manager.cc:194] Registered new tserver with Master: e774593d949d4b81b9bdb3e30ad22f15 (127.7.123.195:36859)
07:43:16.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.832016 8375 heartbeater.cc:511] Master 127.7.123.252:38375 requested a full tablet report, sending...
07:43:16.833 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.832518 7783 ts_manager.cc:194] Registered new tserver with Master: e774593d949d4b81b9bdb3e30ad22f15 (127.7.123.195:36859)
07:43:16.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.833876 7708 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.123.195:45999
07:43:16.834 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.834458 7862 ts_manager.cc:194] Registered new tserver with Master: e774593d949d4b81b9bdb3e30ad22f15 (127.7.123.195:36859)
07:43:16.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:16.845296 7663 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
07:43:16.857 [DEBUG - main] (MiniKuduCluster.java:195) Response:
07:43:16.857 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_masters {
}
07:43:16.859 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_masters {
masters {
id {
type: MASTER
index: 0
}
bound_rpc_address {
host: "127.7.123.254"
port: 37521
}
bound_http_address {
host: "127.7.123.254"
port: 41571
}
}
masters {
id {
type: MASTER
index: 1
}
bound_rpc_address {
host: "127.7.123.253"
port: 41641
}
bound_http_address {
host: "127.7.123.253"
port: 34721
}
}
masters {
id {
type: MASTER
index: 2
}
bound_rpc_address {
host: "127.7.123.252"
port: 38375
}
bound_http_address {
host: "127.7.123.252"
port: 44113
}
}
}
07:43:16.859 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_tservers {
}
07:43:16.860 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_tservers {
tservers {
id {
type: TSERVER
index: 0
}
bound_rpc_address {
host: "127.7.123.193"
port: 41763
}
bound_http_address {
host: "127.7.123.193"
port: 43733
}
}
tservers {
id {
type: TSERVER
index: 1
}
bound_rpc_address {
host: "127.7.123.194"
port: 43897
}
bound_http_address {
host: "127.7.123.194"
port: 45151
}
}
tservers {
id {
type: TSERVER
index: 2
}
bound_rpc_address {
host: "127.7.123.195"
port: 36859
}
bound_http_address {
host: "127.7.123.195"
port: 34727
}
}
}
07:43:16.860 [INFO - main] (KuduTestHarness.java:157) Creating a new Kudu client...
07:43:16.861 [DEBUG - main] (SecurityUtil.java:105) Using ticketCache: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster4024828061966791765/krb5kdc/krb5cc
07:43:16.862 [DEBUG - main] (SecurityUtil.java:122) Could not login via JAAS. Using no credentials: Unable to obtain Principal Name for authentication
07:43:16.863 [INFO - Time-limited test] (TempDirUtils.java:71) Using the temp directory defined by TEST_TMPDIR: /tmp/dist-test-taskNAFIDr/test-tmp
07:43:16.863 [INFO - Time-limited test] (KuduBinaryLocator.java:83) Using Kudu binary directory specified by system property 'kuduBinDir': ../../../build/tsan/bin
07:43:16.864 [INFO - Time-limited test] (MiniKuduCluster.java:217) Starting process: [/tmp/dist-test-taskNAFIDr/java/build/dist-test/../../../build/tsan/bin/kudu, test, mini_cluster, --serialization=pb]
07:43:16.866 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: create_cluster {
num_masters: 3
num_tservers: 0
enable_kerberos: true
cluster_root: "/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147"
extra_master_flags: "--authn_token_validity_seconds=10"
hms_mode: NONE
mini_kdc_options {
ticket_lifetime: "10s"
renew_lifetime: "20s"
}
principal: "kudu"
mini_oidc_options {
}
}
07:43:17.203 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:43:17.203 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_cluster {
}
07:43:17.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:17.206310 8386 mini_oidc.cc:152] Starting JWKS server
07:43:17.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:17.209941 8386 webserver.cc:460] Webserver started at http://127.0.0.1:45213/ using document root <none> and password file <none>
07:43:17.210 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:17.210343 8386 mini_oidc.cc:189] Starting OIDC Discovery server
07:43:17.212 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:17.212026 8386 webserver.cc:460] Webserver started at http://127.0.0.1:40001/ using document root <none> and password file <none>
07:43:17.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Loading random data
07:43:17.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Initializing database '/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/principal' for realm 'KRBTEST.COM',
07:43:17.275 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master key name 'K/M@KRBTEST.COM'
07:43:17.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:17 dist-test-slave-bjlg krb5kdc[8398](info): setting up network...
07:43:17.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
07:43:17.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:17 dist-test-slave-bjlg krb5kdc[8398](info): set up 2 sockets
07:43:17.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:17 dist-test-slave-bjlg krb5kdc[8398](info): commencing operation
07:43:17.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) krb5kdc: starting...
07:43:17.838 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:17.838308 8377 heartbeater.cc:503] Master 127.7.123.254:37521 was elected leader, sending a full tablet report...
07:43:18.318 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.318156 8386 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.002s sys 0.017s
07:43:18.332 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
07:43:18.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
07:43:18.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "test-admin@KRBTEST.COM" created.
07:43:18.359 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
07:43:18.373 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
07:43:18.374 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "test-user@KRBTEST.COM" created.
07:43:18.390 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
07:43:18.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
07:43:18.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "joe-interloper@KRBTEST.COM" created.
07:43:18.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
07:43:18.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/test-user.keytab.
07:43:18.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/test-user.keytab.
07:43:18.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:18 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:18.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Password for test-admin@KRBTEST.COM:
07:43:18.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-06-20T07:43:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
07:43:18.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-06-20T07:43:18Z Disabled control of system clock
07:43:18.491 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for kudu/127.8.48.190@KRBTEST.COM; defaulting to no policy
07:43:18.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
07:43:18.494 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "kudu/127.8.48.190@KRBTEST.COM" created.
07:43:18.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
07:43:18.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.8.48.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.190.keytab.
07:43:18.515 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.8.48.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.190.keytab.
07:43:18.517 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:18.517482 8386 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/wal
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/logs
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data/info.pb
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.48.190:43767
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.48.190
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.190.keytab
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.8.48.190
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.48.148:41955
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:18.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:18.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:18.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:18.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.48.190:43767,127.8.48.189:36719,127.8.48.188:34721
07:43:18.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:18.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45213/jwks.json
07:43:18.519 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10 with env {KRB5CCNAME=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
07:43:18.826 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.826436 8411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:43:18.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.827104 8411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:43:18.828 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.827953 8411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:43:18.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.861340 8411 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=10
07:43:18.861 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.861630 8411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:43:18.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.862033 8411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:43:18.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.862327 8411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:43:18.862 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.862613 8411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:43:18.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.862895 8411 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:43:18.863 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.863116 8411 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:45213/jwks.json
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:18.904940 8411 master_runner.cc:386] Master server non-default flags:
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.48.148:41955
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/wal
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.48.190:43767,127.8.48.189:36719,127.8.48.188:34721
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.190.keytab
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.8.48.190
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.48.190:43767
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45213/jwks.json
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=<redacted>
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:18.905 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data/info.pb
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.48.190
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/logs
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 20 Jun 2026 06:17:27 UTC on ccycloud.quasar-mfmkhc.root.comops.site
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4456919
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:43:18.906 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:18.906384 8411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:43:18.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:18.907418 8411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:43:18.926 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.925976 8418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:20.191 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.926427 8420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:20.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:18.927656 8411 server_base.cc:1034] running on GCE node
07:43:20.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:18.926728 8417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:20.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.195636 8411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:43:20.199 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.198807 8411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:43:20.200 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.200244 8411 hybrid_clock.cc:648] HybridClock initialized: now 1781941400200187 us; error 69 us; skew 500 ppm
07:43:20.201 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.201138 8411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:43:20.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:20 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941400, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:20.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.207139 8411 init.cc:377] Logged in from keytab as kudu/127.8.48.190@KRBTEST.COM (short username kudu)
07:43:20.212 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.212235 8411 webserver.cc:460] Webserver started at http://127.8.48.190:39683/ using document root <none> and password file <none>
07:43:20.213 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.213219 8411 fs_manager.cc:359] Metadata directory not provided
07:43:20.213 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.213454 8411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:43:20.214 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.213925 8411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:43:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.219183 8411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data/instance:
07:43:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "702bd4ab12ab4829adf4a33d8fc0c082"
07:43:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:20 on dist-test-slave-bjlg"
07:43:20.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.220357 8411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/wal/instance:
07:43:20.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "702bd4ab12ab4829adf4a33d8fc0c082"
07:43:20.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:20 on dist-test-slave-bjlg"
07:43:20.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.227728 8411 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
07:43:20.233 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.233474 8429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:43:20.234 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.234463 8411 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.234807 8411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data,/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/wal
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "702bd4ab12ab4829adf4a33d8fc0c082"
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:20 on dist-test-slave-bjlg"
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.235143 8411 fs_report.cc:383] FS layout report
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/wal
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/wal
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data/data
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:43:20.235 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:43:20.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:43:20.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:43:20.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:43:20.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:43:20.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:43:20.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:43:20.272 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.272369 8411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:43:20.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.279575 8411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:43:20.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.280085 8411 kserver.cc:163] Server-wide thread pool size limit: 3276
07:43:20.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.353633 8411 rpc_server.cc:307] RPC server started. Bound to: 127.8.48.190:43767
07:43:20.354 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.353713 8481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.48.190:43767 every 8 connection(s)
07:43:20.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.356407 8411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data/info.pb
07:43:20.358 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.358656 8386 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu as pid 8411
07:43:20.360 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.359158 8386 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/wal/instance
07:43:20.363 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.362629 8482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
07:43:20.376 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for kudu/127.8.48.189@KRBTEST.COM; defaulting to no policy
07:43:20.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.374588 8482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } has no permanent_uuid. Determining permanent_uuid...
07:43:20.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
07:43:20.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "kudu/127.8.48.189@KRBTEST.COM" created.
07:43:20.409 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:20 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941400, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.190@KRBTEST.COM for kudu/127.8.48.190@KRBTEST.COM
07:43:20.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
07:43:20.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.8.48.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.189.keytab.
07:43:20.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.8.48.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.189.keytab.
07:43:20.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.414230 8386 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:20.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:20.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/wal
07:43:20.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data
07:43:20.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:43:20.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:43:20.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:43:20.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:20.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:20.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/logs
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data/info.pb
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.48.189:36719
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.48.189
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.189.keytab
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.8.48.189
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.48.148:41955
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.48.190:43767,127.8.48.189:36719,127.8.48.188:34721
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45213/jwks.json
07:43:20.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10 with env {KRB5CCNAME=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
07:43:20.425 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.425247 8482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } has no permanent_uuid. Determining permanent_uuid...
07:43:20.427 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.427481 8432 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.48.189:36719: connect: Connection refused (error 111)
07:43:20.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.431439 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.189:36719: Network error: Client connection negotiation failed: client connection to 127.8.48.189:36719: connect: Connection refused (error 111)
07:43:20.481 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.481020 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } attempt: 1
07:43:20.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.484805 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.189:36719: Network error: Client connection negotiation failed: client connection to 127.8.48.189:36719: connect: Connection refused (error 111)
07:43:20.553 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.553251 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } attempt: 2
07:43:20.557 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.557057 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.189:36719: Network error: Client connection negotiation failed: client connection to 127.8.48.189:36719: connect: Connection refused (error 111)
07:43:20.649 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.648442 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } attempt: 3
07:43:20.652 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.652307 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.189:36719: Network error: Client connection negotiation failed: client connection to 127.8.48.189:36719: connect: Connection refused (error 111)
07:43:20.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.730746 8488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:43:20.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.731318 8488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:43:20.732 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.731983 8488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:43:20.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.764086 8488 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=10
07:43:20.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.764318 8488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:43:20.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.764608 8488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:43:20.764 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.764806 8488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:43:20.765 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.765005 8488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:43:20.765 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.765190 8488 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:43:20.765 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.765348 8488 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:45213/jwks.json
07:43:20.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.795748 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } attempt: 4
07:43:20.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.799467 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.189:36719: Network error: Client connection negotiation failed: client connection to 127.8.48.189:36719: connect: Connection refused (error 111)
07:43:20.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.806473 8488 master_runner.cc:386] Master server non-default flags:
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.48.148:41955
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/wal
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.48.190:43767,127.8.48.189:36719,127.8.48.188:34721
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.189.keytab
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.8.48.189
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.48.189:36719
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45213/jwks.json
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=<redacted>
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data/info.pb
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.48.189
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/logs
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 20 Jun 2026 06:17:27 UTC on ccycloud.quasar-mfmkhc.root.comops.site
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4456919
07:43:20.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:43:20.808 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.807942 8488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:43:20.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.809159 8488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:43:20.827 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.826864 8495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:21.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:21.098918 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } attempt: 5
07:43:21.103 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:21.102844 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.189:36719: Network error: Client connection negotiation failed: client connection to 127.8.48.189:36719: connect: Connection refused (error 111)
07:43:21.650 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:21.650444 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } attempt: 6
07:43:21.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:21.658048 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.189:36719: Network error: Client connection negotiation failed: client connection to 127.8.48.189:36719: connect: Connection refused (error 111)
07:43:22.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.826923 8498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:22.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:20.827313 8496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:22.099 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:20.828814 8488 server_base.cc:1034] running on GCE node
07:43:22.103 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.103646 8488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:43:22.107 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.106792 8488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:43:22.108 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.108377 8488 hybrid_clock.cc:648] HybridClock initialized: now 1781941402108341 us; error 42 us; skew 500 ppm
07:43:22.109 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.109563 8488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:43:22.115 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:22 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941402, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:22.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.116701 8488 init.cc:377] Logged in from keytab as kudu/127.8.48.189@KRBTEST.COM (short username kudu)
07:43:22.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.122798 8488 webserver.cc:460] Webserver started at http://127.8.48.189:41867/ using document root <none> and password file <none>
07:43:22.123 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.123703 8488 fs_manager.cc:359] Metadata directory not provided
07:43:22.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.123883 8488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:43:22.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.124271 8488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:43:22.129 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.129194 8488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data/instance:
07:43:22.129 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "66a11a4de48941949e660e0a2a385352"
07:43:22.129 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:22 on dist-test-slave-bjlg"
07:43:22.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.130335 8488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/wal/instance:
07:43:22.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "66a11a4de48941949e660e0a2a385352"
07:43:22.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:22 on dist-test-slave-bjlg"
07:43:22.137 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.137400 8488 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
07:43:22.143 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.143061 8510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:43:22.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.144296 8488 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
07:43:22.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.144631 8488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data,/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/wal
07:43:22.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "66a11a4de48941949e660e0a2a385352"
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:22 on dist-test-slave-bjlg"
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.144932 8488 fs_report.cc:383] FS layout report
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/wal
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/wal
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data/data
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:43:22.145 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:43:22.146 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:43:22.146 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:43:22.146 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:43:22.212 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.211862 8488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:43:22.218 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.218361 8488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:43:22.218 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.218788 8488 kserver.cc:163] Server-wide thread pool size limit: 3276
07:43:22.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.291468 8488 rpc_server.cc:307] RPC server started. Bound to: 127.8.48.189:36719
07:43:22.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.291539 8561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.48.189:36719 every 8 connection(s)
07:43:22.294 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.294324 8488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data/info.pb
07:43:22.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.299973 8562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
07:43:22.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.301987 8386 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu as pid 8488
07:43:22.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.302353 8386 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/wal/instance
07:43:22.317 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.314049 8562 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } has no permanent_uuid. Determining permanent_uuid...
07:43:22.320 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for kudu/127.8.48.188@KRBTEST.COM; defaulting to no policy
07:43:22.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
07:43:22.323 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "kudu/127.8.48.188@KRBTEST.COM" created.
07:43:22.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:22 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941402, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.189@KRBTEST.COM for kudu/127.8.48.190@KRBTEST.COM
07:43:22.347 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
07:43:22.347 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.8.48.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.188.keytab.
07:43:22.347 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.8.48.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.188.keytab.
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.349879 8386 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/wal
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/logs
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data/info.pb
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:22.350 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.48.188:34721
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.48.188
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.188.keytab
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.8.48.188
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.48.148:41955
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:22.351 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:22.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.48.190:43767,127.8.48.189:36719,127.8.48.188:34721
07:43:22.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:22.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45213/jwks.json
07:43:22.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10 with env {KRB5CCNAME=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
07:43:22.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.356269 8562 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } has no permanent_uuid. Determining permanent_uuid...
07:43:22.371 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:22 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941402, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.189@KRBTEST.COM for kudu/127.8.48.189@KRBTEST.COM
07:43:22.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.383468 8562 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } has no permanent_uuid. Determining permanent_uuid...
07:43:22.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.385555 8513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:22.402 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.401898 8562 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:22.452 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.451516 8562 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 1
07:43:22.456 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.455721 8562 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:22.524 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.524142 8562 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 2
07:43:22.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.528393 8562 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:22.620 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.619834 8562 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 3
07:43:22.624 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.623679 8562 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:22.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.701884 8568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:43:22.702 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.702407 8568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:43:22.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.703056 8568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:43:22.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.718571 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } attempt: 7
07:43:22.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:22 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941400, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.190@KRBTEST.COM for kudu/127.8.48.189@KRBTEST.COM
07:43:22.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.736819 8568 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=10
07:43:22.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.737080 8568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:43:22.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.737509 8568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:43:22.737 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.737811 8568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:43:22.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.738096 8568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:43:22.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.738396 8568 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:43:22.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.738624 8568 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:45213/jwks.json
07:43:22.742 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.742439 8482 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } has no permanent_uuid. Determining permanent_uuid...
07:43:22.747 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.747385 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:22.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.767091 8562 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 4
07:43:22.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.771410 8562 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:22.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.784128 8568 master_runner.cc:386] Master server non-default flags:
07:43:22.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:22.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.48.148:41955
07:43:22.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:22.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:22.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/wal
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.48.190:43767,127.8.48.189:36719,127.8.48.188:34721
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.188.keytab
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.8.48.188
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.48.188:34721
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45213/jwks.json
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=<redacted>
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data/info.pb
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.48.188
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/logs
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 20 Jun 2026 06:17:27 UTC on ccycloud.quasar-mfmkhc.root.comops.site
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4456919
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:43:22.785 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.785349 8568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:43:22.786 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.786298 8568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:43:22.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.804824 8578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:22.806 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.805778 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 1
07:43:22.810 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.809652 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:22.891 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.891121 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 2
07:43:22.896 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.896078 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:22.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:22.981516 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 3
07:43:22.986 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.986078 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:23.071 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:23.070765 8562 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 5
07:43:23.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:23.075098 8562 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:23.127 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:23.126487 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 4
07:43:23.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:23.130451 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:23.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:23.413846 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 5
07:43:23.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:23.419550 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:23.623 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:23.622578 8562 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 6
07:43:23.630 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:23.629478 8562 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:23.972 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:23.971915 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 6
07:43:23.979 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:23.979198 8482 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.48.188:34721: Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:24.087 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.804992 8581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:24.087 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:22.805303 8579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:24.087 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:24.087427 8580 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
07:43:24.087 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.087556 8568 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
07:43:24.092 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.092123 8568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:43:24.095 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.094779 8568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:43:24.096 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.096185 8568 hybrid_clock.cc:648] HybridClock initialized: now 1781941404096147 us; error 58 us; skew 500 ppm
07:43:24.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.097118 8568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:43:24.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:24 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941404, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:24.103 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.103154 8568 init.cc:377] Logged in from keytab as kudu/127.8.48.188@KRBTEST.COM (short username kudu)
07:43:24.109 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.109163 8568 webserver.cc:460] Webserver started at http://127.8.48.188:36461/ using document root <none> and password file <none>
07:43:24.110 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.110255 8568 fs_manager.cc:359] Metadata directory not provided
07:43:24.110 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.110486 8568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:43:24.111 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.110958 8568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
07:43:24.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.115981 8568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data/instance:
07:43:24.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "d58842f24bc74e8a868c8440ff1bb279"
07:43:24.116 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:24 on dist-test-slave-bjlg"
07:43:24.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.117172 8568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/wal/instance:
07:43:24.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "d58842f24bc74e8a868c8440ff1bb279"
07:43:24.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:24 on dist-test-slave-bjlg"
07:43:24.125 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.125247 8568 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
07:43:24.130 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.130720 8596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:43:24.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.131754 8568 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
07:43:24.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.132108 8568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data,/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/wal
07:43:24.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "d58842f24bc74e8a868c8440ff1bb279"
07:43:24.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:24 on dist-test-slave-bjlg"
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.132450 8568 fs_report.cc:383] FS layout report
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/wal
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/wal
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data/data
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:43:24.133 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:43:24.172 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.172451 8568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:43:24.179 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.178838 8568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:43:24.179 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.179325 8568 kserver.cc:163] Server-wide thread pool size limit: 3276
07:43:24.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.252043 8568 rpc_server.cc:307] RPC server started. Bound to: 127.8.48.188:34721
07:43:24.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.252125 8647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.48.188:34721 every 8 connection(s)
07:43:24.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.254870 8568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data/info.pb
07:43:24.260 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.259944 8648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
07:43:24.261 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.261595 8386 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu as pid 8568
07:43:24.262 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.262091 8386 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/wal/instance
07:43:24.276 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.274185 8648 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } has no permanent_uuid. Determining permanent_uuid...
07:43:24.300 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:24 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941404, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.188@KRBTEST.COM for kudu/127.8.48.190@KRBTEST.COM
07:43:24.304 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:24 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.48.188@KRBTEST.COM
07:43:24.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.312908 8648 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } has no permanent_uuid. Determining permanent_uuid...
07:43:24.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:24 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941404, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.188@KRBTEST.COM for kudu/127.8.48.189@KRBTEST.COM
07:43:24.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:24 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.48.189@KRBTEST.COM
07:43:24.340 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.339831 8648 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } has no permanent_uuid. Determining permanent_uuid...
07:43:24.353 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:24 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941404, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.188@KRBTEST.COM for kudu/127.8.48.188@KRBTEST.COM
07:43:24.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:24 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.48.190@KRBTEST.COM
07:43:24.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.375401 8648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: Bootstrap starting.
07:43:24.379 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.379436 8386 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
07:43:24.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.381614 8648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: Neither blocks nor log segments found. Creating new log.
07:43:24.383 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.383315 8648 log.cc:824] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: Log is configured to *not* fsync() on all Append() calls
07:43:24.387 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:43:24.387 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: get_kdc_env_vars {
}
07:43:24.388 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: get_kdc_env_vars {
env_vars {
key: "KRB5CCNAME"
value: "/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5cc"
}
env_vars {
key: "KRB5_CONFIG"
value: "/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5.conf"
}
env_vars {
key: "KRB5_KDC_PROFILE"
value: "/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kdc.conf"
}
env_vars {
key: "KUDU_ENABLE_KRB5_REALM_FIX"
value: "yes"
}
}
07:43:24.388 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.388212 8648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: No bootstrap required, opened a new log
07:43:24.388 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: get_masters {
}
07:43:24.389 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response: get_masters {
masters {
id {
type: MASTER
index: 0
}
bound_rpc_address {
host: "127.8.48.190"
port: 43767
}
bound_http_address {
host: "127.8.48.190"
port: 39683
}
}
masters {
id {
type: MASTER
index: 1
}
bound_rpc_address {
host: "127.8.48.189"
port: 36719
}
bound_http_address {
host: "127.8.48.189"
port: 41867
}
}
masters {
id {
type: MASTER
index: 2
}
bound_rpc_address {
host: "127.8.48.188"
port: 34721
}
bound_http_address {
host: "127.8.48.188"
port: 36461
}
}
}
07:43:24.390 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: get_tservers {
}
07:43:24.390 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:43:24.390 [INFO - Time-limited test] (MiniKuduCluster.java:622) Running kinit for user test-admin
07:43:24.390 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: kinit {
username: "test-admin"
}
07:43:24.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.414266 8648 raft_consensus.cc:348] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:24.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.414911 8648 raft_consensus.cc:374] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
07:43:24.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.415185 8648 raft_consensus.cc:729] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58842f24bc74e8a868c8440ff1bb279, State: Initialized, Role: FOLLOWER
07:43:24.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.416010 8648 consensus_queue.cc:260] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:24.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:24 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:24.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Password for test-admin@KRBTEST.COM:
07:43:24.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.418884 8659 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:24.420 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.419792 8659 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: This master's current role is: FOLLOWER
07:43:24.420 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.420534 8648 sys_catalog.cc:564] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: configured and running, proceeding with master startup.
07:43:24.426 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:43:24.427 [DEBUG - Time-limited test] (SecurityUtil.java:105) Using ticketCache: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5cc
07:43:24.428 [DEBUG - Time-limited test] (SecurityUtil.java:119) Logged in as subject: test-admin@KRBTEST.COM
07:43:24.428 [DEBUG - Time-limited test] (SecurityUtil.java:105) Using ticketCache: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5cc
07:43:24.429 [DEBUG - Time-limited test] (SecurityUtil.java:119) Logged in as subject: test-admin@KRBTEST.COM
07:43:24.430 [DEBUG - Time-limited test] (SecurityContext.java:161) Using caller-provided subject with Kerberos principal test-admin@KRBTEST.COM. Caller is responsible for refreshing credentials.
07:43:24.434 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:24.434494 8602 tablet.cc:2369] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: Can't schedule compaction. Clean time has not been advanced past its initial value.
07:43:24.446 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:24.445945 8670 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
07:43:24.446 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:24.446312 8670 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
07:43:24.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.442097 8568 master_runner.cc:186] Error getting master registration for 127.8.48.190:43767: OK, instance_id { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" instance_seqno: 1781941400333300 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
07:43:24.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.465444 8568 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
07:43:24.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.690193 8562 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 7
07:43:24.703 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:24 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941402, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.189@KRBTEST.COM for kudu/127.8.48.188@KRBTEST.COM
07:43:24.719 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.719470 8562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: Bootstrap starting.
07:43:24.725 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.724803 8562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: Neither blocks nor log segments found. Creating new log.
07:43:24.726 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.726558 8562 log.cc:824] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: Log is configured to *not* fsync() on all Append() calls
07:43:24.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.730854 8562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: No bootstrap required, opened a new log
07:43:24.752 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.751587 8562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:24.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.752336 8562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
07:43:24.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.752525 8562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a11a4de48941949e660e0a2a385352, State: Initialized, Role: FOLLOWER
07:43:24.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.753288 8562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:24.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.756307 8674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:24.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.757225 8674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: This master's current role is: FOLLOWER
07:43:24.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.758033 8562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: configured and running, proceeding with master startup.
07:43:24.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:24.792675 8685 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
07:43:24.793 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:24.793030 8685 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
07:43:24.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.785673 8488 master_runner.cc:186] Error getting master registration for 127.8.48.190:43767: OK, instance_id { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" instance_seqno: 1781941400333300 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
07:43:24.821 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.820883 8488 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
07:43:24.913 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.913125 8659 raft_consensus.cc:482] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
07:43:24.914 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.913663 8659 raft_consensus.cc:504] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:24.917 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.916049 8659 leader_election.cc:288] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 702bd4ab12ab4829adf4a33d8fc0c082 (127.8.48.190:43767), 66a11a4de48941949e660e0a2a385352 (127.8.48.189:36719)
07:43:24.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.917562 8537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d58842f24bc74e8a868c8440ff1bb279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a11a4de48941949e660e0a2a385352" is_pre_election: true
07:43:24.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.918455 8537 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58842f24bc74e8a868c8440ff1bb279 in term 0.
07:43:24.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.919687 8599 leader_election.cc:302] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a11a4de48941949e660e0a2a385352, d58842f24bc74e8a868c8440ff1bb279; no voters:
07:43:24.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.920624 8659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 0 FOLLOWER]: Leader pre-election won for term 1
07:43:24.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.921288 8659 raft_consensus.cc:482] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
07:43:24.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.922013 8659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 0 FOLLOWER]: Advancing to term 1
07:43:24.934 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.933202 8659 raft_consensus.cc:504] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:24.938 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.937458 8659 leader_election.cc:288] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [CANDIDATE]: Term 1 election: Requested vote from peers 702bd4ab12ab4829adf4a33d8fc0c082 (127.8.48.190:43767), 66a11a4de48941949e660e0a2a385352 (127.8.48.189:36719)
07:43:24.939 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.938786 8537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d58842f24bc74e8a868c8440ff1bb279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a11a4de48941949e660e0a2a385352"
07:43:24.943 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.939339 8537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 0 FOLLOWER]: Advancing to term 1
07:43:24.950 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.950198 8537 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58842f24bc74e8a868c8440ff1bb279 in term 1.
07:43:24.952 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.951587 8599 leader_election.cc:302] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66a11a4de48941949e660e0a2a385352, d58842f24bc74e8a868c8440ff1bb279; no voters:
07:43:24.952 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.952402 8659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 1 FOLLOWER]: Leader election won for term 1
07:43:24.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.954531 8659 raft_consensus.cc:686] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 1 LEADER]: Becoming Leader. State: Replica: d58842f24bc74e8a868c8440ff1bb279, State: Running, Role: LEADER
07:43:24.956 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.955610 8659 consensus_queue.cc:237] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:24.973 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.972726 8687 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d58842f24bc74e8a868c8440ff1bb279. Latest consensus state: current_term: 1 leader_uuid: "d58842f24bc74e8a868c8440ff1bb279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:24.973 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.973497 8687 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: This master's current role is: LEADER
07:43:24.977 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:24.976351 8516 tablet.cc:2369] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: Can't schedule compaction. Clean time has not been advanced past its initial value.
07:43:24.977 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.976684 8689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
07:43:24.984 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:24.983932 8689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
07:43:25.006 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.005434 8537 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 1 FOLLOWER]: Refusing update from remote peer d58842f24bc74e8a868c8440ff1bb279: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
07:43:25.008 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.007344 8687 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
07:43:25.013 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.012609 8482 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } attempt: 7
07:43:25.042 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.042096 8674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d58842f24bc74e8a868c8440ff1bb279. Latest consensus state: current_term: 1 leader_uuid: "d58842f24bc74e8a868c8440ff1bb279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:25.043 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.043251 8674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: This master's current role is: FOLLOWER
07:43:25.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.061982 8659 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d58842f24bc74e8a868c8440ff1bb279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:25.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.062741 8659 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: This master's current role is: LEADER
07:43:25.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.064284 8689 catalog_manager.cc:1347] Generated new cluster ID: 29938e88a6024259b0a8e797fbca4bbc
07:43:25.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.064574 8689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
07:43:25.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:25 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941400, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.190@KRBTEST.COM for kudu/127.8.48.188@KRBTEST.COM
07:43:25.079 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.078703 8674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d58842f24bc74e8a868c8440ff1bb279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:25.079 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.079452 8674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: This master's current role is: FOLLOWER
07:43:25.102 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.101584 8689 catalog_manager.cc:1370] Generated new certificate authority record
07:43:25.113 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.112668 8689 catalog_manager.cc:1504] Loading token signing keys...
07:43:25.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.121954 8482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: Bootstrap starting.
07:43:25.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.135711 8482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: Neither blocks nor log segments found. Creating new log.
07:43:25.138 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.138348 8482 log.cc:824] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: Log is configured to *not* fsync() on all Append() calls
07:43:25.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.154994 8689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: Generated new TSK 0
07:43:25.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.157152 8689 catalog_manager.cc:1514] Initializing in-progress tserver states...
07:43:25.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.166153 8482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: No bootstrap required, opened a new log
07:43:25.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.206424 8482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:25.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.207515 8482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
07:43:25.208 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.207841 8482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 702bd4ab12ab4829adf4a33d8fc0c082, State: Initialized, Role: FOLLOWER
07:43:25.209 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.208798 8482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:25.215 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.213284 8697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:25.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.215533 8482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: configured and running, proceeding with master startup.
07:43:25.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.215695 8456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d58842f24bc74e8a868c8440ff1bb279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "702bd4ab12ab4829adf4a33d8fc0c082"
07:43:25.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.216441 8456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [term 0 FOLLOWER]: Advancing to term 1
07:43:25.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.221122 8457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d58842f24bc74e8a868c8440ff1bb279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" is_pre_election: true
07:43:25.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.227902 8456 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58842f24bc74e8a868c8440ff1bb279 in term 1.
07:43:25.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.229583 8455 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [term 1 FOLLOWER]: Refusing update from remote peer d58842f24bc74e8a868c8440ff1bb279: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
07:43:25.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.246565 8659 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [LEADER]: Connected to new peer: Peer: permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
07:43:25.259 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.257591 8697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: This master's current role is: FOLLOWER
07:43:25.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:25.301746 8644 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.8.48.188:34721
07:43:25.306 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:25.305781 8436 tablet.cc:2369] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: Can't schedule compaction. Clean time has not been advanced past its initial value.
07:43:25.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.310853 8705 mvcc.cc:205] Tried to move back new op lower bound from 7298831994879565824 to 7298831994707976192. Current Snapshot: MvccSnapshot[applied={T|T < 7298831994879565824}]
07:43:25.332 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.331043 8697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d58842f24bc74e8a868c8440ff1bb279. Latest consensus state: current_term: 1 leader_uuid: "d58842f24bc74e8a868c8440ff1bb279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:25.332 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.331838 8697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: This master's current role is: FOLLOWER
07:43:25.332 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.331507 8703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d58842f24bc74e8a868c8440ff1bb279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:25.332 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.332281 8703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: This master's current role is: FOLLOWER
07:43:25.356 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.355767 8714 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
07:43:25.713 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.712706 8659 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d58842f24bc74e8a868c8440ff1bb279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:25.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.714672 8659 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: This master's current role is: LEADER
07:43:25.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.798782 8685 catalog_manager.cc:1259] Loaded cluster ID: 29938e88a6024259b0a8e797fbca4bbc
07:43:25.799 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.799054 8685 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: loading cluster ID for follower catalog manager: success
07:43:25.807 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.806730 8685 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: acquiring CA information for follower catalog manager: success
07:43:25.814 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:25.813862 8685 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:43:26.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:26.370048 8714 catalog_manager.cc:1259] Loaded cluster ID: 29938e88a6024259b0a8e797fbca4bbc
07:43:26.370 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:26.370604 8714 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: loading cluster ID for follower catalog manager: success
07:43:26.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:26.379700 8714 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: acquiring CA information for follower catalog manager: success
07:43:26.386 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:26.386536 8714 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:43:27.431 [INFO - Time-limited test] (MiniKuduCluster.java:399) Killing master server 127.8.48.189:36719
07:43:27.431 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
id {
type: MASTER
index: 1
}
}
07:43:27.432 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:27.432094 8386 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu with pid 8488
07:43:27.462 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:43:27.462 [INFO - Time-limited test] (MiniKuduCluster.java:399) Killing master server 127.8.48.188:34721
07:43:27.463 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
id {
type: MASTER
index: 2
}
}
07:43:27.463 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:27.463379 8386 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu with pid 8568
07:43:27.503 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:43:27.504 [INFO - Time-limited test] (MiniKuduCluster.java:399) Killing master server 127.8.48.190:43767
07:43:27.504 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: stop_daemon {
id {
type: MASTER
index: 0
}
}
07:43:27.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:27.504879 8386 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu with pid 8411
07:43:27.527 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:43:27.527 [INFO - Time-limited test] (MiniKuduCluster.java:380) Starting master server 127.8.48.189:36719
07:43:27.527 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
id {
type: MASTER
index: 1
}
}
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:27.528978 8386 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/wal
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/logs
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data/info.pb
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:27.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.48.189:36719
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.48.189
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=41867
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.189.keytab
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.8.48.189
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.48.148:41955
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.48.190:43767,127.8.48.189:36719,127.8.48.188:34721
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45213/jwks.json
07:43:27.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10 with env {KRB5CCNAME=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
07:43:27.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.857168 8722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:43:27.857 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.857765 8722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:43:27.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.858366 8722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:43:27.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.907245 8722 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=10
07:43:27.907 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.907547 8722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:43:27.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.907991 8722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:43:27.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.908334 8722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:43:27.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.908658 8722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:43:27.908 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.908867 8722 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:43:27.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.909010 8722 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:45213/jwks.json
07:43:27.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:27.946386 8722 master_runner.cc:386] Master server non-default flags:
07:43:27.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:27.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.48.148:41955
07:43:27.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:27.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:27.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/wal
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.48.190:43767,127.8.48.189:36719,127.8.48.188:34721
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.189.keytab
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.8.48.189
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.48.189:36719
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45213/jwks.json
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=<redacted>
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data/info.pb
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.48.189
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=41867
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/logs
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 20 Jun 2026 06:17:27 UTC on ccycloud.quasar-mfmkhc.root.comops.site
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4456919
07:43:27.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:43:27.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:27.947479 8722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:43:27.948 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:27.948381 8722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:43:27.971 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.969933 8729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:29.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.970021 8728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:29.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:27.970925 8731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:29.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.228101 8730 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
07:43:29.228 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.228237 8722 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
07:43:29.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.232472 8722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:43:29.236 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.236028 8722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:43:29.237 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.237534 8722 hybrid_clock.cc:648] HybridClock initialized: now 1781941409237488 us; error 64 us; skew 500 ppm
07:43:29.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.238490 8722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:43:29.249 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:29 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941409, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:29.250 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.250362 8722 init.cc:377] Logged in from keytab as kudu/127.8.48.189@KRBTEST.COM (short username kudu)
07:43:29.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.255358 8722 webserver.cc:460] Webserver started at http://127.8.48.189:41867/ using document root <none> and password file <none>
07:43:29.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.256238 8722 fs_manager.cc:359] Metadata directory not provided
07:43:29.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.256438 8722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:43:29.264 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.264375 8722 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
07:43:29.269 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.268891 8742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:43:29.270 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.270047 8722 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
07:43:29.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.270342 8722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data,/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/wal
07:43:29.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "66a11a4de48941949e660e0a2a385352"
07:43:29.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:22 on dist-test-slave-bjlg"
07:43:29.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.273133 8722 fs_report.cc:383] FS layout report
07:43:29.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:43:29.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/wal
07:43:29.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/wal
07:43:29.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data/data
07:43:29.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:43:29.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:43:29.273 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:43:29.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:43:29.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:43:29.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:43:29.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:43:29.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:43:29.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:43:29.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:43:29.274 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:43:29.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.312055 8722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:43:29.318 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.318210 8722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:43:29.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.318876 8722 kserver.cc:163] Server-wide thread pool size limit: 3276
07:43:29.394 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.394533 8722 rpc_server.cc:307] RPC server started. Bound to: 127.8.48.189:36719
07:43:29.395 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.394613 8793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.48.189:36719 every 8 connection(s)
07:43:29.397 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.397728 8722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-1/data/info.pb
07:43:29.408 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.407732 8386 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu as pid 8722
07:43:29.408 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:43:29.408 [INFO - Time-limited test] (MiniKuduCluster.java:380) Starting master server 127.8.48.188:34721
07:43:29.408 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
id {
type: MASTER
index: 2
}
}
07:43:29.408 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.408675 8794 sys_catalog.cc:263] Verifying existing consensus state
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.409891 8386 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/wal
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/logs
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data/info.pb
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.48.188:34721
07:43:29.410 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.48.188
07:43:29.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36461
07:43:29.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.188.keytab
07:43:29.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.8.48.188
07:43:29.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:43:29.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
07:43:29.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
07:43:29.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.48.148:41955
07:43:29.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:29.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:29.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:29.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:29.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.48.190:43767,127.8.48.189:36719,127.8.48.188:34721
07:43:29.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:29.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45213/jwks.json
07:43:29.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10 with env {KRB5CCNAME=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
07:43:29.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.414873 8794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: Bootstrap starting.
07:43:29.453 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.452869 8794 log.cc:824] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: Log is configured to *not* fsync() on all Append() calls
07:43:29.468 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.468297 8794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
07:43:29.469 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.469092 8794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: Bootstrap complete.
07:43:29.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.493805 8794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:29.496 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.496264 8794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a11a4de48941949e660e0a2a385352, State: Initialized, Role: FOLLOWER
07:43:29.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.497311 8794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:29.501 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.500317 8798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:29.501 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.501180 8798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: This master's current role is: FOLLOWER
07:43:29.502 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.501967 8794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: configured and running, proceeding with master startup.
07:43:29.534 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.534435 8809 catalog_manager.cc:1259] Loaded cluster ID: 29938e88a6024259b0a8e797fbca4bbc
07:43:29.534 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.534694 8809 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: loading cluster ID for follower catalog manager: success
07:43:29.543 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.543174 8809 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: acquiring CA information for follower catalog manager: success
07:43:29.549 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.549276 8809 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:43:29.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.529480 8722 master_runner.cc:186] Error getting master registration for 127.8.48.190:43767: Network error: Client connection negotiation failed: client connection to 127.8.48.190:43767: connect: Connection refused (error 111),
07:43:29.556 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.556231 8722 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
07:43:29.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.794607 8796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:43:29.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.795266 8796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:43:29.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.795861 8796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:43:29.828 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.828639 8796 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=10
07:43:29.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.828929 8796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:43:29.829 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.829206 8796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:43:29.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.829454 8796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:43:29.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.829741 8796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:43:29.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.830026 8796 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:43:29.830 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.830227 8796 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:45213/jwks.json
07:43:29.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.876232 8796 master_runner.cc:386] Master server non-default flags:
07:43:29.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:29.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.48.148:41955
07:43:29.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:29.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:29.876 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/wal
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.48.190:43767,127.8.48.189:36719,127.8.48.188:34721
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.188.keytab
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.8.48.188
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.48.188:34721
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45213/jwks.json
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=<redacted>
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data/info.pb
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.48.188
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=36461
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/logs
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 20 Jun 2026 06:17:27 UTC on ccycloud.quasar-mfmkhc.root.comops.site
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4456919
07:43:29.877 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:43:29.878 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.877866 8796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:43:29.879 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:29.879194 8796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:43:29.911 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.910668 8817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:31.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.334329 8824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
07:43:31.336 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.335253 8824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:31.345 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.345466 8824 leader_election.cc:288] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 702bd4ab12ab4829adf4a33d8fc0c082 (127.8.48.190:43767), d58842f24bc74e8a868c8440ff1bb279 (127.8.48.188:34721)
07:43:31.366 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:31.365674 8745 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.48.190:43767: connect: Connection refused (error 111)
07:43:31.377 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:31.377511 8745 leader_election.cc:334] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 702bd4ab12ab4829adf4a33d8fc0c082 (127.8.48.190:43767): Network error: Client connection negotiation failed: client connection to 127.8.48.190:43767: connect: Connection refused (error 111)
07:43:31.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:31.380410 8745 leader_election.cc:334] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d58842f24bc74e8a868c8440ff1bb279 (127.8.48.188:34721): Network error: Client connection negotiation failed: client connection to 127.8.48.188:34721: connect: Connection refused (error 111)
07:43:31.381 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.381001 8745 leader_election.cc:302] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 66a11a4de48941949e660e0a2a385352; no voters: 702bd4ab12ab4829adf4a33d8fc0c082, d58842f24bc74e8a868c8440ff1bb279
07:43:31.382 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.381894 8824 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
07:43:31.455 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:31.297271 8816 debug-util.cc:398] Leaking SignalData structure 0x7b0800014920 after lost signal to thread 8796
07:43:31.456 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:31.455897 8796 thread.cc:636] GCE (cloud detector) Time spent creating pthread: real 1.547s user 0.649s sys 0.872s
07:43:31.456 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:31.456295 8796 thread.cc:602] GCE (cloud detector) Time spent starting thread: real 1.547s user 0.649s sys 0.872s
07:43:31.456 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:29.912364 8818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:31.465 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:31.465063 8829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:31.466 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.466005 8796 server_base.cc:1034] running on GCE node
07:43:31.467 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.467298 8796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:43:31.470 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.470309 8796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:43:31.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.471750 8796 hybrid_clock.cc:648] HybridClock initialized: now 1781941411471685 us; error 71 us; skew 500 ppm
07:43:31.472 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.472506 8796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:43:31.478 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:31 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941411, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:31.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.479202 8796 init.cc:377] Logged in from keytab as kudu/127.8.48.188@KRBTEST.COM (short username kudu)
07:43:31.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.485322 8796 webserver.cc:460] Webserver started at http://127.8.48.188:36461/ using document root <none> and password file <none>
07:43:31.486 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.486479 8796 fs_manager.cc:359] Metadata directory not provided
07:43:31.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.486829 8796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:43:31.498 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.497836 8796 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.005s sys 0.001s
07:43:31.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.504314 8835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:43:31.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.505769 8796 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.002s
07:43:31.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.506246 8796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data,/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/wal
07:43:31.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "d58842f24bc74e8a868c8440ff1bb279"
07:43:31.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:24 on dist-test-slave-bjlg"
07:43:31.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.510023 8796 fs_report.cc:383] FS layout report
07:43:31.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:43:31.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/wal
07:43:31.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/wal
07:43:31.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data/data
07:43:31.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:43:31.511 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:43:31.511 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:43:31.511 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:43:31.511 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:43:31.511 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:43:31.511 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:43:31.511 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:43:31.511 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:43:31.511 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:43:31.511 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:43:31.589 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.589006 8796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:43:31.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.598320 8796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:43:31.599 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.598784 8796 kserver.cc:163] Server-wide thread pool size limit: 3276
07:43:31.690 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.690274 8796 rpc_server.cc:307] RPC server started. Bound to: 127.8.48.188:34721
07:43:31.691 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.690706 8886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.48.188:34721 every 8 connection(s)
07:43:31.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.694264 8796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-2/data/info.pb
07:43:31.704 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.703471 8386 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu as pid 8796
07:43:31.704 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:43:31.704 [INFO - Time-limited test] (MiniKuduCluster.java:380) Starting master server 127.8.48.190:43767
07:43:31.704 [DEBUG - Time-limited test] (MiniKuduCluster.java:185) Request: start_daemon {
id {
type: MASTER
index: 0
}
}
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.705988 8386 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/wal
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/logs
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data/info.pb
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
07:43:31.706 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.48.190:43767
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.48.190
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39683
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.190.keytab
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.8.48.190
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.48.148:41955
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.48.190:43767,127.8.48.189:36719,127.8.48.188:34721
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45213/jwks.json
07:43:31.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10 with env {KRB5CCNAME=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
07:43:31.712 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.711755 8887 sys_catalog.cc:263] Verifying existing consensus state
07:43:31.720 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.720541 8887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: Bootstrap starting.
07:43:31.775 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.770628 8887 log.cc:824] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: Log is configured to *not* fsync() on all Append() calls
07:43:31.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.797015 8887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
07:43:31.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.798076 8887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: Bootstrap complete.
07:43:31.844 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.840418 8887 raft_consensus.cc:348] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:31.845 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.844792 8887 raft_consensus.cc:729] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58842f24bc74e8a868c8440ff1bb279, State: Initialized, Role: FOLLOWER
07:43:31.847 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.846166 8887 consensus_queue.cc:260] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:31.851 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.850565 8891 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:31.851 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.851572 8891 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: This master's current role is: FOLLOWER
07:43:31.853 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.852788 8887 sys_catalog.cc:564] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: configured and running, proceeding with master startup.
07:43:31.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.947480 8902 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
07:43:31.987 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.956220 8796 master_runner.cc:186] Error getting master registration for 127.8.48.190:43767: Network error: Client connection negotiation failed: client connection to 127.8.48.190:43767: connect: Connection refused (error 111),
07:43:31.987 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:31.987576 8796 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
07:43:32.120 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.120340 8889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
07:43:32.121 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.121115 8889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
07:43:32.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.121869 8889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
07:43:32.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.155364 8889 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=10
07:43:32.155 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.155642 8889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
07:43:32.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.156045 8889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
07:43:32.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.156392 8889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
07:43:32.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.156705 8889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
07:43:32.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.156992 8889 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
07:43:32.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.157218 8889 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:45213/jwks.json
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:32.194962 8889 master_runner.cc:386] Master server non-default flags:
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.8.48.148:41955
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/wal
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --authn_token_validity_seconds=10
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.8.48.190:43767,127.8.48.189:36719,127.8.48.188:34721
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/krb5kdc/kudu_127.8.48.190.keytab
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.8.48.190
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.8.48.190:43767
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:45213/jwks.json
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=<redacted>
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data/info.pb
07:43:32.195 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.8.48.190
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=39683
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/logs
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.18.0-SNAPSHOT
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by systest at 20 Jun 2026 06:17:27 UTC on ccycloud.quasar-mfmkhc.root.comops.site
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 4456919
07:43:32.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
07:43:32.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:32.197435 8889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
07:43:32.198 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:32.198578 8889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
07:43:32.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.219364 8910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:32.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.693508 8883 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.8.48.188:34721
07:43:32.951 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:32.950587 8915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
07:43:32.951 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:32.951333 8915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:32.955 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:32.955179 8915 leader_election.cc:288] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 702bd4ab12ab4829adf4a33d8fc0c082 (127.8.48.190:43767), d58842f24bc74e8a868c8440ff1bb279 (127.8.48.188:34721)
07:43:32.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:32.961160 8902 catalog_manager.cc:1259] Loaded cluster ID: 29938e88a6024259b0a8e797fbca4bbc
07:43:32.961 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:32.961529 8902 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: loading cluster ID for follower catalog manager: success
07:43:32.976 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.975910 8745 leader_election.cc:334] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 702bd4ab12ab4829adf4a33d8fc0c082 (127.8.48.190:43767): Network error: Client connection negotiation failed: client connection to 127.8.48.190:43767: connect: Connection refused (error 111)
07:43:32.981 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:32.981410 8902 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: acquiring CA information for follower catalog manager: success
07:43:32.987 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:32.987571 8902 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:43:33.002 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:33 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941409, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.189@KRBTEST.COM for kudu/127.8.48.188@KRBTEST.COM
07:43:33.029 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.028375 8862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "66a11a4de48941949e660e0a2a385352" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d58842f24bc74e8a868c8440ff1bb279" is_pre_election: true
07:43:33.030 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.029882 8862 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a11a4de48941949e660e0a2a385352 in term 1.
07:43:33.032 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.032196 8745 leader_election.cc:302] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66a11a4de48941949e660e0a2a385352, d58842f24bc74e8a868c8440ff1bb279; no voters: 702bd4ab12ab4829adf4a33d8fc0c082
07:43:33.034 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.033823 8915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 1 FOLLOWER]: Leader pre-election won for term 2
07:43:33.034 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.034318 8915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
07:43:33.035 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.034891 8915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 1 FOLLOWER]: Advancing to term 2
07:43:33.051 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.050050 8915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:33.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.052934 8915 leader_election.cc:288] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [CANDIDATE]: Term 2 election: Requested vote from peers 702bd4ab12ab4829adf4a33d8fc0c082 (127.8.48.190:43767), d58842f24bc74e8a868c8440ff1bb279 (127.8.48.188:34721)
07:43:33.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.055141 8862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "66a11a4de48941949e660e0a2a385352" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d58842f24bc74e8a868c8440ff1bb279"
07:43:33.056 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.055686 8862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 1 FOLLOWER]: Advancing to term 2
07:43:33.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:33.057992 8745 leader_election.cc:334] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 702bd4ab12ab4829adf4a33d8fc0c082 (127.8.48.190:43767): Network error: Client connection negotiation failed: client connection to 127.8.48.190:43767: connect: Connection refused (error 111)
07:43:33.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.066123 8862 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a11a4de48941949e660e0a2a385352 in term 2.
07:43:33.069 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.067368 8745 leader_election.cc:302] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66a11a4de48941949e660e0a2a385352, d58842f24bc74e8a868c8440ff1bb279; no voters: 702bd4ab12ab4829adf4a33d8fc0c082
07:43:33.070 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.069828 8915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 2 FOLLOWER]: Leader election won for term 2
07:43:33.074 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.073619 8915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [term 2 LEADER]: Becoming Leader. State: Replica: 66a11a4de48941949e660e0a2a385352, State: Running, Role: LEADER
07:43:33.075 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.074958 8915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:33.089 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.087774 8921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66a11a4de48941949e660e0a2a385352. Latest consensus state: current_term: 2 leader_uuid: "66a11a4de48941949e660e0a2a385352" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:33.089 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.089078 8921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: This master's current role is: LEADER
07:43:33.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.098364 8923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
07:43:33.111 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.111315 8923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
07:43:33.117 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.117537 8923 catalog_manager.cc:1259] Loaded cluster ID: 29938e88a6024259b0a8e797fbca4bbc
07:43:33.118 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.117949 8923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
07:43:33.126 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.126103 8923 catalog_manager.cc:1504] Loading token signing keys...
07:43:33.132 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.132180 8923 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352: Loaded TSK: 0
07:43:33.135 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.134783 8923 catalog_manager.cc:1514] Initializing in-progress tserver states...
07:43:33.512 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.219916 8912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:33.512 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:32.219934 8909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
07:43:33.513 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:33.512653 8911 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
07:43:33.513 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.512789 8889 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
07:43:33.518 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.517905 8889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
07:43:33.521 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.521392 8889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
07:43:33.523 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.522881 8889 hybrid_clock.cc:648] HybridClock initialized: now 1781941413522806 us; error 85 us; skew 500 ppm
07:43:33.523 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.523721 8889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
07:43:33.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:33 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:33.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.530162 8889 init.cc:377] Logged in from keytab as kudu/127.8.48.190@KRBTEST.COM (short username kudu)
07:43:33.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.536801 8889 webserver.cc:460] Webserver started at http://127.8.48.190:39683/ using document root <none> and password file <none>
07:43:33.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.537983 8889 fs_manager.cc:359] Metadata directory not provided
07:43:33.538 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.538277 8889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
07:43:33.548 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.547807 8889 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
07:43:33.554 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.553922 8931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
07:43:33.555 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.555200 8889 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
07:43:33.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.555593 8889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data,/tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/wal
07:43:33.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "702bd4ab12ab4829adf4a33d8fc0c082"
07:43:33.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-20 07:43:20 on dist-test-slave-bjlg"
07:43:33.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.558075 8889 fs_report.cc:383] FS layout report
07:43:33.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
07:43:33.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/wal
07:43:33.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/wal
07:43:33.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data/data
07:43:33.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
07:43:33.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
07:43:33.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
07:43:33.558 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
07:43:33.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
07:43:33.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
07:43:33.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
07:43:33.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
07:43:33.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
07:43:33.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
07:43:33.559 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
07:43:33.610 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.609858 8889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
07:43:33.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.616170 8889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
07:43:33.616 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.616711 8889 kserver.cc:163] Server-wide thread pool size limit: 3276
07:43:33.724 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.723857 8862 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [term 2 FOLLOWER]: Refusing update from remote peer 66a11a4de48941949e660e0a2a385352: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
07:43:33.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.726262 8961 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
07:43:33.734 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:33.733886 8745 consensus_peers.cc:487] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 -> Peer 702bd4ab12ab4829adf4a33d8fc0c082 (127.8.48.190:43767): Couldn't send request to peer 702bd4ab12ab4829adf4a33d8fc0c082. Status: Network error: Client connection negotiation failed: client connection to 127.8.48.190:43767: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
07:43:33.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.758117 8968 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66a11a4de48941949e660e0a2a385352. Latest consensus state: current_term: 2 leader_uuid: "66a11a4de48941949e660e0a2a385352" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:33.759 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.759143 8968 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: This master's current role is: FOLLOWER
07:43:33.776 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.768615 8961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "66a11a4de48941949e660e0a2a385352" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:33.777 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.769383 8961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: This master's current role is: LEADER
07:43:33.783 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.782963 8968 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "66a11a4de48941949e660e0a2a385352" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:33.784 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.783982 8968 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58842f24bc74e8a868c8440ff1bb279 [sys.catalog]: This master's current role is: FOLLOWER
07:43:33.888 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.888504 8992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.48.190:43767 every 8 connection(s)
07:43:33.889 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.889130 8889 rpc_server.cc:307] RPC server started. Bound to: 127.8.48.190:43767
07:43:33.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.893003 8889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNAFIDr/test-tmp/mini-kudu-cluster5598069476477745147/master-0/data/info.pb
07:43:33.895 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.894517 8386 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu as pid 8889
07:43:33.895 [DEBUG - Time-limited test] (MiniKuduCluster.java:195) Response:
07:43:33.909 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:33.909 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.909369 8993 sys_catalog.cc:263] Verifying existing consensus state
07:43:33.910 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.8.48.188:34721(127.8.48.188:34721)] connecting to peer
07:43:33.912 [DEBUG - Time-limited test] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:33.924 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:33.927 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:33.933 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:33.917754 8993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: Bootstrap starting.
07:43:33.935 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.188:34721(127.8.48.188:34721)] Successfully connected to peer
07:43:33.941 [WARN - kudu-nio-0] (SecurityContext.java:209) Caller-provided Subject has a Kerberos ticket that is about to expire. Kudu expects the application to renew or re-acquire its own tickets before expiration.
07:43:33.942 [DEBUG - kudu-nio-0] (Negotiator.java:529) SASL mechanism GSSAPI chosen for peer 127.8.48.189
07:43:33.948 [DEBUG - kudu-nio-1] (Negotiator.java:529) SASL mechanism GSSAPI chosen for peer 127.8.48.188
07:43:33.955 [DEBUG - kudu-nio-0] (Negotiator.java:529) SASL mechanism GSSAPI chosen for peer 127.8.48.190
07:43:33.991 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:33 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.48.188@KRBTEST.COM
07:43:34.009 [DEBUG - kudu-nio-1] (Negotiator.java:919) Authenticated connection [id: 0x5220747d, L:/127.0.0.1:47434 - R:/127.8.48.188:34721] using SASL/GSSAPI
07:43:34.019 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: Server requires Kerberos, but this client is not authenticated (missing or expired TGT)
07:43:34.019 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: Server requires Kerberos, but this client is not authenticated (missing or expired TGT)
07:43:34.021 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: Server requires Kerberos, but this client is not authenticated (missing or expired TGT)
07:43:34.022 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: Server requires Kerberos, but this client is not authenticated (missing or expired TGT)
07:43:34.022 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:34.023 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:34.023 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.023368 8999 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.922183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41616 (local address 127.8.48.190:43767)
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.946443 (+ 24260us) server_negotiation.cc:206] Beginning negotiation
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.946479 (+ 36us) server_negotiation.cc:399] Waiting for connection header
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.946548 (+ 69us) server_negotiation.cc:407] Connection header received
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.953256 (+ 6708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.953318 (+ 62us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.953802 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.954355 (+ 553us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.973138 (+ 18783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.983961 (+ 10823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.015861 (+ 31900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
07:43:34.023 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.017279 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
07:43:34.024 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.017597 (+ 318us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
07:43:34.024 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.021291 (+ 3694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41616: BlockingRecv error: failed to read from TLS socket (remote: 127.0.0.1:41616): Cannot send after transport endpoint shutdown (error 108)
07:43:34.024 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":23719,"thread_start_us":26696,"threads_started":1}
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.025310 8998 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.921722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37716 (local address 127.8.48.189:36719)
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.937898 (+ 16176us) server_negotiation.cc:206] Beginning negotiation
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.937937 (+ 39us) server_negotiation.cc:399] Waiting for connection header
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.937991 (+ 54us) server_negotiation.cc:407] Connection header received
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.938146 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.938173 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.938642 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.938879 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.965613 (+ 26734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:33.967749 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.005464 (+ 37715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
07:43:34.025 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.006502 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
07:43:34.026 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.006768 (+ 266us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
07:43:34.026 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.023636 (+ 16868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37716: BlockingRecv error: failed to read from TLS socket (remote: 127.0.0.1:37716): Cannot send after transport endpoint shutdown (error 108)
07:43:34.026 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":15606,"thread_start_us":17636,"threads_started":1}
07:43:34.026 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.026444 8993 log.cc:824] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: Log is configured to *not* fsync() on all Append() calls
07:43:34.030 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:34.031 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:34.031 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:34.032 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:34.034 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.034 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.034 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.034 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.035 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:34.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.036407 8998 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.032959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37722 (local address 127.8.48.189:36719)
07:43:34.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.033473 (+ 514us) server_negotiation.cc:206] Beginning negotiation
07:43:34.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.033498 (+ 25us) server_negotiation.cc:399] Waiting for connection header
07:43:34.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.033540 (+ 42us) server_negotiation.cc:407] Connection header received
07:43:34.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.033671 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.033697 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.033901 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.034054 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.036 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.035325 (+ 1271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37722: BlockingRecv error: recv got EOF from 127.0.0.1:37722 (error 108)
07:43:34.037 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":118}
07:43:34.037 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:34.037 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.037307 8999 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.037 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.032866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41632 (local address 127.8.48.190:43767)
07:43:34.037 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.033384 (+ 518us) server_negotiation.cc:206] Beginning negotiation
07:43:34.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.033412 (+ 28us) server_negotiation.cc:399] Waiting for connection header
07:43:34.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.033460 (+ 48us) server_negotiation.cc:407] Connection header received
07:43:34.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.033591 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.033623 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.033857 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.034027 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.035727 (+ 1700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41632: BlockingRecv error: recv got EOF from 127.0.0.1:41632 (error 108)
07:43:34.038 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":132}
07:43:34.039 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:34.049 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.049365 8993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
07:43:34.050 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:34.050 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:34.053 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:34.053 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.050462 8993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: Bootstrap complete.
07:43:34.055 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.055 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.056777 8999 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.052178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41638 (local address 127.8.48.190:43767)
07:43:34.057 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:34.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.052713 (+ 535us) server_negotiation.cc:206] Beginning negotiation
07:43:34.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.052737 (+ 24us) server_negotiation.cc:399] Waiting for connection header
07:43:34.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.053299 (+ 562us) server_negotiation.cc:407] Connection header received
07:43:34.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.053775 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.053816 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.054035 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.054183 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.055825 (+ 1642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41638: BlockingRecv error: recv got EOF from 127.0.0.1:41638 (error 108)
07:43:34.057 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":117}
07:43:34.058 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:34.060 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.060 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.060 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:34.061 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:34.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.062269 8998 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.057270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37732 (local address 127.8.48.189:36719)
07:43:34.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.058117 (+ 847us) server_negotiation.cc:206] Beginning negotiation
07:43:34.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.058149 (+ 32us) server_negotiation.cc:399] Waiting for connection header
07:43:34.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.059063 (+ 914us) server_negotiation.cc:407] Connection header received
07:43:34.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.059221 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.059249 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.059486 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.059649 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.061640 (+ 1991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37732: BlockingRecv error: recv got EOF from 127.0.0.1:37732 (error 108)
07:43:34.062 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":414}
07:43:34.069 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:34.070 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:34.079 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:34.081 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:34.081 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.081 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.082 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.082768 8999 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.076973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.8.48.190:43767)
07:43:34.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.077494 (+ 521us) server_negotiation.cc:206] Beginning negotiation
07:43:34.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.077520 (+ 26us) server_negotiation.cc:399] Waiting for connection header
07:43:34.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.079315 (+ 1795us) server_negotiation.cc:407] Connection header received
07:43:34.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.079820 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.079850 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.080063 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.080244 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.081829 (+ 1585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41640: BlockingRecv error: recv got EOF from 127.0.0.1:41640 (error 108)
07:43:34.083 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":149}
07:43:34.083 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.083 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:34.083 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:34.085 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.085178 8998 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.073920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37738 (local address 127.8.48.189:36719)
07:43:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.080422 (+ 6502us) server_negotiation.cc:206] Beginning negotiation
07:43:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.080447 (+ 25us) server_negotiation.cc:399] Waiting for connection header
07:43:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.081259 (+ 812us) server_negotiation.cc:407] Connection header received
07:43:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.081669 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.081703 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.081929 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.082071 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.084290 (+ 2219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37738: BlockingRecv error: recv got EOF from 127.0.0.1:37738 (error 108)
07:43:34.085 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":6072}
07:43:34.087 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.084220 8993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:34.088 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.087670 8993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 702bd4ab12ab4829adf4a33d8fc0c082, State: Initialized, Role: FOLLOWER
07:43:34.089 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.088662 8993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } }
07:43:34.091 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.091063 9002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:34.092 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.091933 9002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: This master's current role is: FOLLOWER
07:43:34.093 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.093235 8993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: configured and running, proceeding with master startup.
07:43:34.110 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:34.111 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:34.111 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:34.112 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:34.115 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.115 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.115 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:34.116 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.117 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.117 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:34.117 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:34.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.121842 8999 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.115361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41656 (local address 127.8.48.190:43767)
07:43:34.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.115858 (+ 497us) server_negotiation.cc:206] Beginning negotiation
07:43:34.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.115882 (+ 24us) server_negotiation.cc:399] Waiting for connection header
07:43:34.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.115927 (+ 45us) server_negotiation.cc:407] Connection header received
07:43:34.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.116076 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.116115 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.116298 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.116486 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.121188 (+ 4702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41656: BlockingRecv error: recv got EOF from 127.0.0.1:41656 (error 108)
07:43:34.122 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":121}
07:43:34.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.123854 8998 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.113507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37752 (local address 127.8.48.189:36719)
07:43:34.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.114223 (+ 716us) server_negotiation.cc:206] Beginning negotiation
07:43:34.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.114248 (+ 25us) server_negotiation.cc:399] Waiting for connection header
07:43:34.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.114291 (+ 43us) server_negotiation.cc:407] Connection header received
07:43:34.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.114422 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.114438 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.114604 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.114790 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.122829 (+ 8039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37752: BlockingRecv error: recv got EOF from 127.0.0.1:37752 (error 108)
07:43:34.124 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":304}
07:43:34.129 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:34.130 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:34.131 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:34.137 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:34.141 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.141 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.142 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:34.143 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.143105 9013 catalog_manager.cc:1259] Loaded cluster ID: 29938e88a6024259b0a8e797fbca4bbc
07:43:34.143 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.143383 9013 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: loading cluster ID for follower catalog manager: success
07:43:34.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.144186 8998 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.133016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37764 (local address 127.8.48.189:36719)
07:43:34.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.136259 (+ 3243us) server_negotiation.cc:206] Beginning negotiation
07:43:34.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.136284 (+ 25us) server_negotiation.cc:399] Waiting for connection header
07:43:34.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.137797 (+ 1513us) server_negotiation.cc:407] Connection header received
07:43:34.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.138727 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.138770 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.139176 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.139418 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.143033 (+ 3615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37764: BlockingRecv error: recv got EOF from 127.0.0.1:37764 (error 108)
07:43:34.144 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":2849}
07:43:34.146 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.147 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.147 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:34.147 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:34.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.150410 9016 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.133394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41672 (local address 127.8.48.190:43767)
07:43:34.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.145663 (+ 12269us) server_negotiation.cc:206] Beginning negotiation
07:43:34.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.145692 (+ 29us) server_negotiation.cc:399] Waiting for connection header
07:43:34.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.145743 (+ 51us) server_negotiation.cc:407] Connection header received
07:43:34.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.145902 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.145936 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.146161 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.148071 (+ 1910us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.148975 (+ 904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41672: BlockingRecv error: recv got EOF from 127.0.0.1:41672 (error 108)
07:43:34.150 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":11797,"thread_start_us":7544,"threads_started":1}
07:43:34.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.163923 9013 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: acquiring CA information for follower catalog manager: success
07:43:34.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:34 dist-test-slave-bjlg krb5kdc[8398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.190@KRBTEST.COM for kudu/127.8.48.190@KRBTEST.COM
07:43:34.169 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:34.170 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:34.174 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:34.175 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:34.177 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.176152 9013 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
07:43:34.182 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.182 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.182 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:34.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.183872 8998 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.178189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37778 (local address 127.8.48.189:36719)
07:43:34.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.178803 (+ 614us) server_negotiation.cc:206] Beginning negotiation
07:43:34.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.178827 (+ 24us) server_negotiation.cc:399] Waiting for connection header
07:43:34.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.178869 (+ 42us) server_negotiation.cc:407] Connection header received
07:43:34.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.181245 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.181279 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.181511 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.181700 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.182948 (+ 1248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37778: BlockingRecv error: recv got EOF from 127.0.0.1:37778 (error 108)
07:43:34.184 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":172}
07:43:34.186 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.186 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.186 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:34.186 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:34.190 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:34.190 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:34.191 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:34.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.195672 8958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [term 1 FOLLOWER]: Advancing to term 2
07:43:34.196 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:34.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.196658 8999 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.179541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.8.48.190:43767)
07:43:34.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.184837 (+ 5296us) server_negotiation.cc:206] Beginning negotiation
07:43:34.196 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.184873 (+ 36us) server_negotiation.cc:399] Waiting for connection header
07:43:34.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.184930 (+ 57us) server_negotiation.cc:407] Connection header received
07:43:34.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.185076 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.185116 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.185416 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.185649 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.195511 (+ 9862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41696: BlockingRecv error: recv got EOF from 127.0.0.1:41696 (error 108)
07:43:34.197 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1628,"threads_started":1}
07:43:34.197 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.197 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.198 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:34.198 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.200 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.200 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:34.200 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:34.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.202170 9016 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.192293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41710 (local address 127.8.48.190:43767)
07:43:34.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.195731 (+ 3438us) server_negotiation.cc:206] Beginning negotiation
07:43:34.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.195754 (+ 23us) server_negotiation.cc:399] Waiting for connection header
07:43:34.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.195795 (+ 41us) server_negotiation.cc:407] Connection header received
07:43:34.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.195918 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.195940 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.196347 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.200327 (+ 3980us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.202 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.201136 (+ 809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41710: BlockingRecv error: recv got EOF from 127.0.0.1:41710 (error 108)
07:43:34.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":257}
07:43:34.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.202615 8998 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.191649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37786 (local address 127.8.48.189:36719)
07:43:34.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.195655 (+ 4006us) server_negotiation.cc:206] Beginning negotiation
07:43:34.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.195690 (+ 35us) server_negotiation.cc:399] Waiting for connection header
07:43:34.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.197335 (+ 1645us) server_negotiation.cc:407] Connection header received
07:43:34.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.197484 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.197509 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.197733 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.197902 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.201364 (+ 3462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37786: BlockingRecv error: recv got EOF from 127.0.0.1:37786 (error 108)
07:43:34.203 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":3588}
07:43:34.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.225721 9020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "66a11a4de48941949e660e0a2a385352" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:34.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.226773 9020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: This master's current role is: FOLLOWER
07:43:34.234 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.233450 9002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66a11a4de48941949e660e0a2a385352. Latest consensus state: current_term: 2 leader_uuid: "66a11a4de48941949e660e0a2a385352" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:34.234 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.234210 9002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 702bd4ab12ab4829adf4a33d8fc0c082 [sys.catalog]: This master's current role is: FOLLOWER
07:43:34.350 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:34.350 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:34.351 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:34.352 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:34.357 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.357 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.358 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:34.358 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.359 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.359 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:34.359 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:34.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.361122 8998 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.352610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37792 (local address 127.8.48.189:36719)
07:43:34.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.356233 (+ 3623us) server_negotiation.cc:206] Beginning negotiation
07:43:34.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.356256 (+ 23us) server_negotiation.cc:399] Waiting for connection header
07:43:34.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.356313 (+ 57us) server_negotiation.cc:407] Connection header received
07:43:34.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.356454 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.356482 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.356747 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.359415 (+ 2668us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.361 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.360111 (+ 696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37792: BlockingRecv error: recv got EOF from 127.0.0.1:37792 (error 108)
07:43:34.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":3184}
07:43:34.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.361171 9016 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.356987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41712 (local address 127.8.48.190:43767)
07:43:34.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.357467 (+ 480us) server_negotiation.cc:206] Beginning negotiation
07:43:34.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.357488 (+ 21us) server_negotiation.cc:399] Waiting for connection header
07:43:34.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.357536 (+ 48us) server_negotiation.cc:407] Connection header received
07:43:34.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.357667 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.357696 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.357908 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.358080 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.360153 (+ 2073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41712: BlockingRecv error: recv got EOF from 127.0.0.1:41712 (error 108)
07:43:34.362 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":80}
07:43:34.569 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:34.570 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:34.572 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:34.573 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:34.577 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.577 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.577 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:34.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.580260 9016 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.574082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41726 (local address 127.8.48.190:43767)
07:43:34.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.574620 (+ 538us) server_negotiation.cc:206] Beginning negotiation
07:43:34.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.574642 (+ 22us) server_negotiation.cc:399] Waiting for connection header
07:43:34.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.574691 (+ 49us) server_negotiation.cc:407] Connection header received
07:43:34.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.574817 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.574842 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.575055 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.575219 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.579289 (+ 4070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41726: BlockingRecv error: recv got EOF from 127.0.0.1:41726 (error 108)
07:43:34.580 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":125}
07:43:34.589 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.589 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:34.590 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:34.590 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:34.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.593166 8998 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:34.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.576160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37806 (local address 127.8.48.189:36719)
07:43:34.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.587952 (+ 11792us) server_negotiation.cc:206] Beginning negotiation
07:43:34.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.587976 (+ 24us) server_negotiation.cc:399] Waiting for connection header
07:43:34.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.588022 (+ 46us) server_negotiation.cc:407] Connection header received
07:43:34.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.588140 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:34.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.588176 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:34.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.588856 (+ 680us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:34.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.591339 (+ 2483us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:34.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:34.591982 (+ 643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37806: BlockingRecv error: recv got EOF from 127.0.0.1:37806 (error 108)
07:43:34.593 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":4120,"thread_start_us":14872,"threads_started":1}
07:43:34.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.646521 8962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "66a11a4de48941949e660e0a2a385352" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "702bd4ab12ab4829adf4a33d8fc0c082" member_type: VOTER last_known_addr { host: "127.8.48.190" port: 43767 } } peers { permanent_uuid: "66a11a4de48941949e660e0a2a385352" member_type: VOTER last_known_addr { host: "127.8.48.189" port: 36719 } } peers { permanent_uuid: "d58842f24bc74e8a868c8440ff1bb279" member_type: VOTER last_known_addr { host: "127.8.48.188" port: 34721 } } }
07:43:34.647 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:34.647326 8962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a11a4de48941949e660e0a2a385352 [sys.catalog]: This master's current role is: LEADER
07:43:34.867 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.866634 8090 debug-util.cc:398] Leaking SignalData structure 0x7b08000ba660 after lost signal to thread 7965
07:43:34.867 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:34.867686 8090 debug-util.cc:398] Leaking SignalData structure 0x7b08000b3e80 after lost signal to thread 8093
07:43:35.129 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:35.130 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:35.130 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:35.131 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:35.163 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:35.163 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:35.164 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:35.169 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:35.170 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:35.170 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:35.170 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:35.173 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:35.173629 9025 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:35.173 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.132736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41734 (local address 127.8.48.190:43767)
07:43:35.173 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.161306 (+ 28570us) server_negotiation.cc:206] Beginning negotiation
07:43:35.174 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.161332 (+ 26us) server_negotiation.cc:399] Waiting for connection header
07:43:35.174 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.161626 (+ 294us) server_negotiation.cc:407] Connection header received
07:43:35.174 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.161772 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:35.174 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.161799 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:35.174 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.162008 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:35.174 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.171845 (+ 9837us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:35.174 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.172519 (+ 674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41734: BlockingRecv error: recv got EOF from 127.0.0.1:41734 (error 108)
07:43:35.174 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":28144,"thread_start_us":36229,"threads_started":1}
07:43:35.175 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:35.175601 9026 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:35.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.135408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37818 (local address 127.8.48.189:36719)
07:43:35.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.159801 (+ 24393us) server_negotiation.cc:206] Beginning negotiation
07:43:35.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.159829 (+ 28us) server_negotiation.cc:399] Waiting for connection header
07:43:35.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.159884 (+ 55us) server_negotiation.cc:407] Connection header received
07:43:35.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.160047 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:35.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.160072 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:35.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.162504 (+ 2432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:35.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.168425 (+ 5921us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:35.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:35.171688 (+ 3263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37818: BlockingRecv error: recv got EOF from 127.0.0.1:37818 (error 108)
07:43:35.176 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":23927,"thread_start_us":24780,"threads_started":1}
07:43:36.310 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:36.310 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:36.311 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:36.311 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:36.328 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:36.328 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:36.328 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:36.331 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:36.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:36.331043 9031 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:36.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.312895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41748 (local address 127.8.48.190:43767)
07:43:36.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.326944 (+ 14049us) server_negotiation.cc:206] Beginning negotiation
07:43:36.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.326974 (+ 30us) server_negotiation.cc:399] Waiting for connection header
07:43:36.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.327025 (+ 51us) server_negotiation.cc:407] Connection header received
07:43:36.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.327164 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:36.331 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:36.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.327189 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:36.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.327419 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:36.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.329151 (+ 1732us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:36.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.329858 (+ 707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41748: BlockingRecv error: recv got EOF from 127.0.0.1:41748 (error 108)
07:43:36.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":13621,"thread_start_us":14717,"threads_started":1}
07:43:36.331 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:36.332 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:36.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:36.333917 9032 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:36.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.313129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37830 (local address 127.8.48.189:36719)
07:43:36.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.330046 (+ 16917us) server_negotiation.cc:206] Beginning negotiation
07:43:36.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.330072 (+ 26us) server_negotiation.cc:399] Waiting for connection header
07:43:36.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.330128 (+ 56us) server_negotiation.cc:407] Connection header received
07:43:36.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.330281 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:36.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.330317 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:36.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.330561 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:36.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.332150 (+ 1589us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:36.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.332843 (+ 693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37830: BlockingRecv error: recv got EOF from 127.0.0.1:37830 (error 108)
07:43:36.334 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":16419,"thread_start_us":16113,"threads_started":1}
07:43:36.750 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:36.750 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:36.751 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:36.751 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:36.754 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:36.754 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:36.754 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:36.754 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:36.754 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:36.755 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:36.755 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:36.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:36.756271 9031 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:36.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.752188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41760 (local address 127.8.48.190:43767)
07:43:36.756 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.752789 (+ 601us) server_negotiation.cc:206] Beginning negotiation
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.752813 (+ 24us) server_negotiation.cc:399] Waiting for connection header
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.752857 (+ 44us) server_negotiation.cc:407] Connection header received
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.752991 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.753015 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.753286 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.753462 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.755411 (+ 1949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41760: BlockingRecv error: recv got EOF from 127.0.0.1:41760 (error 108)
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":159}
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:36.757125 9032 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.752898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37842 (local address 127.8.48.189:36719)
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.753427 (+ 529us) server_negotiation.cc:206] Beginning negotiation
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.753453 (+ 26us) server_negotiation.cc:399] Waiting for connection header
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.753499 (+ 46us) server_negotiation.cc:407] Connection header received
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.753625 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.753652 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.753881 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.754040 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:36.756056 (+ 2016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37842: BlockingRecv error: recv got EOF from 127.0.0.1:37842 (error 108)
07:43:36.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":122}
07:43:38.649 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:38.650 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:38.651 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:38.651 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:38.654 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:38.655 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:38.655 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:38.655 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:38.655 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:38.655 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:38.656 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:38.656876 9040 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.652279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41782 (local address 127.8.48.190:43767)
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.653595 (+ 1316us) server_negotiation.cc:206] Beginning negotiation
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.653628 (+ 33us) server_negotiation.cc:399] Waiting for connection header
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.653680 (+ 52us) server_negotiation.cc:407] Connection header received
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.653851 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.653881 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.654132 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.654342 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.655740 (+ 1398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41782: BlockingRecv error: recv got EOF from 127.0.0.1:41782 (error 108)
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":603,"threads_started":1}
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:38.657735 9041 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.652159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37864 (local address 127.8.48.189:36719)
07:43:38.657 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.654188 (+ 2029us) server_negotiation.cc:206] Beginning negotiation
07:43:38.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.654216 (+ 28us) server_negotiation.cc:399] Waiting for connection header
07:43:38.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.654305 (+ 89us) server_negotiation.cc:407] Connection header received
07:43:38.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.654516 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:38.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.654542 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:38.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.654804 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:38.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.654951 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:38.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:38.656542 (+ 1591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37864: BlockingRecv error: recv got EOF from 127.0.0.1:37864 (error 108)
07:43:38.658 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1502,"thread_start_us":1142,"threads_started":1}
07:43:39.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:39 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941419, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:39.254 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:39.254452 8738 init.cc:295] Successfully reacquired a new kerberos TGT
07:43:41.482 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:41 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941421, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:41.483 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:41.482955 8831 init.cc:295] Successfully reacquired a new kerberos TGT
07:43:42.090 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:42.090 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:42.091 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:42.091 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:42.094 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:42.095 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:42.095 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:42.095 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:42.095 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:42.096 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:42.096 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:42.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:42.097131 9046 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:42.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.091718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.8.48.190:43767)
07:43:42.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.093621 (+ 1903us) server_negotiation.cc:206] Beginning negotiation
07:43:42.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.093646 (+ 25us) server_negotiation.cc:399] Waiting for connection header
07:43:42.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.093695 (+ 49us) server_negotiation.cc:407] Connection header received
07:43:42.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.093843 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:42.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.093870 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:42.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.094090 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:42.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.094278 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:42.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.095997 (+ 1719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41798: BlockingRecv error: recv got EOF from 127.0.0.1:41798 (error 108)
07:43:42.097 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1292,"thread_start_us":1023,"threads_started":1}
07:43:42.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:42.098119 9047 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:42.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.092847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37876 (local address 127.8.48.189:36719)
07:43:42.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.094233 (+ 1386us) server_negotiation.cc:206] Beginning negotiation
07:43:42.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.094262 (+ 29us) server_negotiation.cc:399] Waiting for connection header
07:43:42.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.094365 (+ 103us) server_negotiation.cc:407] Connection header received
07:43:42.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.094568 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:42.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.094593 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:42.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.094813 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:42.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.094985 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:42.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:42.096776 (+ 1791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37876: BlockingRecv error: recv got EOF from 127.0.0.1:37876 (error 108)
07:43:42.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":999,"thread_start_us":703,"threads_started":1}
07:43:43.534 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:43 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941423, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:43.535 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:43.534734 8927 init.cc:295] Successfully reacquired a new kerberos TGT
07:43:45.749 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:45.750 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:45.751 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:45.751 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:45.755 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:45.755 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:45.755 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:45.755 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:45.755 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:45.755 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:45.756 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:45.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:45.757305 9053 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:45.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.751493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.8.48.189:36719)
07:43:45.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754350 (+ 2857us) server_negotiation.cc:206] Beginning negotiation
07:43:45.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754376 (+ 26us) server_negotiation.cc:399] Waiting for connection header
07:43:45.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754415 (+ 39us) server_negotiation.cc:407] Connection header received
07:43:45.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754529 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:45.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754546 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:45.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754772 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:45.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754965 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:45.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.756144 (+ 1179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33982: BlockingRecv error: recv got EOF from 127.0.0.1:33982 (error 108)
07:43:45.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":2408,"thread_start_us":2105,"threads_started":1}
07:43:45.757 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:45.757375 9054 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:45.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.752406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38940 (local address 127.8.48.190:43767)
07:43:45.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754110 (+ 1704us) server_negotiation.cc:206] Beginning negotiation
07:43:45.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754137 (+ 27us) server_negotiation.cc:399] Waiting for connection header
07:43:45.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754202 (+ 65us) server_negotiation.cc:407] Connection header received
07:43:45.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754404 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:45.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754427 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:45.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754739 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:45.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.754944 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:45.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:45.756406 (+ 1462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38940: BlockingRecv error: recv got EOF from 127.0.0.1:38940 (error 108)
07:43:45.758 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1144,"thread_start_us":650,"threads_started":1}
07:43:46.611 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:46.611495 8883 debug-util.cc:398] Leaking SignalData structure 0x7b0800098380 after lost signal to thread 8812
07:43:46.612 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:46.612000 8883 debug-util.cc:398] Leaking SignalData structure 0x7b08000abee0 after lost signal to thread 8886
07:43:48.389 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:48.390 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:48.390 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:48.391 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:48.394 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:48.395 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:48.395 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:48.395 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:48.396 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:48.396 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:48.396 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:48.397 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:48.397608 9060 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:48.397 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.392074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38948 (local address 127.8.48.190:43767)
07:43:48.397 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.393736 (+ 1662us) server_negotiation.cc:206] Beginning negotiation
07:43:48.397 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.393799 (+ 63us) server_negotiation.cc:399] Waiting for connection header
07:43:48.398 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.393854 (+ 55us) server_negotiation.cc:407] Connection header received
07:43:48.398 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.393983 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:48.398 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.394001 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:48.398 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.394186 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:48.398 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.394384 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:48.398 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.396443 (+ 2059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38948: BlockingRecv error: recv got EOF from 127.0.0.1:38948 (error 108)
07:43:48.398 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":987,"thread_start_us":1022,"threads_started":1}
07:43:48.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:48.398671 9061 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:48.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.392157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33996 (local address 127.8.48.189:36719)
07:43:48.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.394304 (+ 2147us) server_negotiation.cc:206] Beginning negotiation
07:43:48.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.394335 (+ 31us) server_negotiation.cc:399] Waiting for connection header
07:43:48.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.394406 (+ 71us) server_negotiation.cc:407] Connection header received
07:43:48.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.394749 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:48.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.394783 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:48.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.395143 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:48.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.395394 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:48.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:48.397118 (+ 1724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33996: BlockingRecv error: recv got EOF from 127.0.0.1:33996 (error 108)
07:43:48.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1448,"thread_start_us":999,"threads_started":1}
07:43:49.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:49 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941429, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:49.257 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:49.256896 8738 init.cc:295] Successfully reacquired a new kerberos TGT
07:43:49.810 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:49.811 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:49.811 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:49.811 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:49.815 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:49.815 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:49.815 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:49.815 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:49.815 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:49.815 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:49.816 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:49.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:49.816984 9064 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:49.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.812809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38952 (local address 127.8.48.190:43767)
07:43:49.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.814456 (+ 1647us) server_negotiation.cc:206] Beginning negotiation
07:43:49.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.814471 (+ 15us) server_negotiation.cc:399] Waiting for connection header
07:43:49.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.814505 (+ 34us) server_negotiation.cc:407] Connection header received
07:43:49.817 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.814609 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.814625 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.814832 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.815007 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.816133 (+ 1126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38952: BlockingRecv error: recv got EOF from 127.0.0.1:38952 (error 108)
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1292,"thread_start_us":1026,"threads_started":1}
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:49.818326 9063 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.812671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34006 (local address 127.8.48.189:36719)
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.814072 (+ 1401us) server_negotiation.cc:206] Beginning negotiation
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.814095 (+ 23us) server_negotiation.cc:399] Waiting for connection header
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.814150 (+ 55us) server_negotiation.cc:407] Connection header received
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.814332 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.814362 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.814610 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.816540 (+ 1930us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:49.817218 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34006: BlockingRecv error: recv got EOF from 127.0.0.1:34006 (error 108)
07:43:49.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":889,"thread_start_us":579,"threads_started":1}
07:43:50.307 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:50.306595 8883 debug-util.cc:398] Leaking SignalData structure 0x7b08000baf60 after lost signal to thread 8812
07:43:50.307 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:50.307567 8883 debug-util.cc:398] Leaking SignalData structure 0x7b08000bd980 after lost signal to thread 8886
07:43:50.630 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:50.630 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:50.631 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:50.631 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:50.633 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:50.634 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:50.634 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:50.634 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:50.634 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:50.634 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:50.635 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:50.635746 9066 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.631618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38958 (local address 127.8.48.190:43767)
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.632875 (+ 1257us) server_negotiation.cc:206] Beginning negotiation
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.632890 (+ 15us) server_negotiation.cc:399] Waiting for connection header
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.632925 (+ 35us) server_negotiation.cc:407] Connection header received
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.633031 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.633047 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.633203 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.633388 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.634669 (+ 1281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38958: BlockingRecv error: recv got EOF from 127.0.0.1:38958 (error 108)
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":892,"thread_start_us":608,"threads_started":1}
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:50.636251 9067 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:50.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.632261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34022 (local address 127.8.48.189:36719)
07:43:50.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.633388 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
07:43:50.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.633416 (+ 28us) server_negotiation.cc:399] Waiting for connection header
07:43:50.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.633463 (+ 47us) server_negotiation.cc:407] Connection header received
07:43:50.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.633595 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:50.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.633619 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:50.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.633850 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:50.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.634028 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:50.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:50.635447 (+ 1419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34022: BlockingRecv error: recv got EOF from 127.0.0.1:34022 (error 108)
07:43:50.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":1656,"threads_started":1}
07:43:51.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:51 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941431, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:51.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:51.485589 8831 init.cc:295] Successfully reacquired a new kerberos TGT
07:43:53.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:53 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941433, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:53.537 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:53.537616 8927 init.cc:295] Successfully reacquired a new kerberos TGT
07:43:53.569 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:53.570 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:53.571 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:53.571 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:53.575 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:53.575 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:53.575 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:53.575 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:53.576 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:53.576 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:53.576 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:53.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:53.577421 9073 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:53.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.572036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.8.48.189:36719)
07:43:53.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574237 (+ 2201us) server_negotiation.cc:206] Beginning negotiation
07:43:53.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574279 (+ 42us) server_negotiation.cc:399] Waiting for connection header
07:43:53.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574332 (+ 53us) server_negotiation.cc:407] Connection header received
07:43:53.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574450 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:53.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574490 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:53.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574681 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:53.577 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574862 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.576211 (+ 1349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56064: BlockingRecv error: recv got EOF from 127.0.0.1:56064 (error 108)
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1867,"thread_start_us":1600,"threads_started":1}
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:53.577787 9074 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.572279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37992 (local address 127.8.48.190:43767)
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574422 (+ 2143us) server_negotiation.cc:206] Beginning negotiation
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574442 (+ 20us) server_negotiation.cc:399] Waiting for connection header
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574510 (+ 68us) server_negotiation.cc:407] Connection header received
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574709 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574737 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.574926 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.575129 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:53.576709 (+ 1580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37992: BlockingRecv error: recv got EOF from 127.0.0.1:37992 (error 108)
07:43:53.578 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1714,"thread_start_us":1395,"threads_started":1}
07:43:55.729 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:55.730 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:55.731 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:55.731 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:55.738 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:55.738 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:55.738 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:55.739 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:55.739 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:55.739 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:55.739 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:55.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:55.740571 9080 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:55.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.732425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56076 (local address 127.8.48.189:36719)
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.737352 (+ 4927us) server_negotiation.cc:206] Beginning negotiation
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.737368 (+ 16us) server_negotiation.cc:399] Waiting for connection header
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.737407 (+ 39us) server_negotiation.cc:407] Connection header received
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.737520 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.737536 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.737749 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.738028 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.739393 (+ 1365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56076: BlockingRecv error: recv got EOF from 127.0.0.1:56076 (error 108)
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":4441,"thread_start_us":4104,"threads_started":1}
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:55.741048 9079 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.732188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38002 (local address 127.8.48.190:43767)
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.737685 (+ 5497us) server_negotiation.cc:206] Beginning negotiation
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.737707 (+ 22us) server_negotiation.cc:399] Waiting for connection header
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.737755 (+ 48us) server_negotiation.cc:407] Connection header received
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.737891 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.737915 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.738304 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.738626 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:55.739927 (+ 1301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38002: BlockingRecv error: recv got EOF from 127.0.0.1:38002 (error 108)
07:43:55.741 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":5067,"thread_start_us":4803,"threads_started":1}
07:43:58.149 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:43:58.150 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:43:58.151 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:43:58.151 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:43:58.155 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:58.155 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:58.155 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:43:58.156 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:58.156 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:43:58.156 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:43:58.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:58.156654 9085 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:58.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.152125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38014 (local address 127.8.48.190:43767)
07:43:58.156 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.154120 (+ 1995us) server_negotiation.cc:206] Beginning negotiation
07:43:58.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.154135 (+ 15us) server_negotiation.cc:399] Waiting for connection header
07:43:58.157 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:43:58.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.154179 (+ 44us) server_negotiation.cc:407] Connection header received
07:43:58.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.154312 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:58.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.154328 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:58.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.154484 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:58.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.154723 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:58.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.155871 (+ 1148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38014: BlockingRecv error: recv got EOF from 127.0.0.1:38014 (error 108)
07:43:58.157 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1476,"thread_start_us":1145,"threads_started":1}
07:43:58.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:43:58.159370 9086 negotiation.cc:337] Failed RPC negotiation. Trace:
07:43:58.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.152134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.8.48.189:36719)
07:43:58.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.154695 (+ 2561us) server_negotiation.cc:206] Beginning negotiation
07:43:58.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.154725 (+ 30us) server_negotiation.cc:399] Waiting for connection header
07:43:58.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.154804 (+ 79us) server_negotiation.cc:407] Connection header received
07:43:58.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.155071 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:43:58.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.155099 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:43:58.159 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.155405 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:43:58.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.157205 (+ 1800us) server_negotiation.cc:226] Negotiated authn=SASL
07:43:58.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:43:58.158082 (+ 877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56092: BlockingRecv error: recv got EOF from 127.0.0.1:56092 (error 108)
07:43:58.160 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1983,"thread_start_us":1596,"threads_started":1}
07:43:59.258 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:43:59 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941439, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:43:59.259 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:43:59.259339 8738 init.cc:295] Successfully reacquired a new kerberos TGT
07:44:00.949 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:44:00.950 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:44:00.950 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:44:00.950 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:44:00.955 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:44:00.955 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:44:00.955 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:44:00.956 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:44:00.956 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:44:00.956 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:44:00.956 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:44:00.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:44:00.957346 9090 negotiation.cc:337] Failed RPC negotiation. Trace:
07:44:00.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.951697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38030 (local address 127.8.48.190:43767)
07:44:00.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.954236 (+ 2539us) server_negotiation.cc:206] Beginning negotiation
07:44:00.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.954256 (+ 20us) server_negotiation.cc:399] Waiting for connection header
07:44:00.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.954306 (+ 50us) server_negotiation.cc:407] Connection header received
07:44:00.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.954498 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:44:00.957 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.954524 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.954739 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.955831 (+ 1092us) server_negotiation.cc:226] Negotiated authn=SASL
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.956398 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38030: BlockingRecv error: recv got EOF from 127.0.0.1:38030 (error 108)
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":2122,"thread_start_us":1855,"threads_started":1}
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:44:00.958073 9089 negotiation.cc:337] Failed RPC negotiation. Trace:
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.951698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56094 (local address 127.8.48.189:36719)
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.954575 (+ 2877us) server_negotiation.cc:206] Beginning negotiation
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.954598 (+ 23us) server_negotiation.cc:399] Waiting for connection header
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.954645 (+ 47us) server_negotiation.cc:407] Connection header received
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.954783 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.954808 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.955029 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.955753 (+ 724us) server_negotiation.cc:226] Negotiated authn=SASL
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:00.956877 (+ 1124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56094: BlockingRecv error: recv got EOF from 127.0.0.1:56094 (error 108)
07:44:00.958 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":2407,"thread_start_us":1991,"threads_started":1}
07:44:01.487 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:44:01 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941441, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:44:01.488 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:44:01.488253 8831 init.cc:295] Successfully reacquired a new kerberos TGT
07:44:03.539 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 20 07:44:03 dist-test-slave-bjlg krb5kdc[8398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781941443, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.48.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
07:44:03.540 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:44:03.540165 8927 init.cc:295] Successfully reacquired a new kerberos TGT
07:44:03.629 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:44:03.630 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:44:03.631 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:44:03.631 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:44:03.634 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:44:03.635 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:44:03.635 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:44:03.635 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:44:03.635 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:44:03.635 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:44:03.636 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:44:03.636767 9094 negotiation.cc:337] Failed RPC negotiation. Trace:
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.632185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52514 (local address 127.8.48.189:36719)
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.633643 (+ 1458us) server_negotiation.cc:206] Beginning negotiation
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.633668 (+ 25us) server_negotiation.cc:399] Waiting for connection header
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.633717 (+ 49us) server_negotiation.cc:407] Connection header received
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.633859 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.633887 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.634110 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.634331 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.635772 (+ 1441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52514: BlockingRecv error: recv got EOF from 127.0.0.1:52514 (error 108)
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":937,"thread_start_us":656,"threads_started":1}
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:44:03.637679 9095 negotiation.cc:337] Failed RPC negotiation. Trace:
07:44:03.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.632188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52622 (local address 127.8.48.190:43767)
07:44:03.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.634135 (+ 1947us) server_negotiation.cc:206] Beginning negotiation
07:44:03.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.634163 (+ 28us) server_negotiation.cc:399] Waiting for connection header
07:44:03.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.634256 (+ 93us) server_negotiation.cc:407] Connection header received
07:44:03.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.634493 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:44:03.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.634519 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:44:03.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.634749 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:44:03.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.634951 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
07:44:03.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.636432 (+ 1481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52622: BlockingRecv error: recv got EOF from 127.0.0.1:52622 (error 108)
07:44:03.638 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":1453,"thread_start_us":1129,"threads_started":1}
07:44:03.810 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.189:36719(127.8.48.189:36719)] connecting to peer
07:44:03.810 [DEBUG - pool-70-thread-1] (Connection.java:748) [peer master-127.8.48.190:43767(127.8.48.190:43767)] connecting to peer
07:44:03.811 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.8.48.189:36719(127.8.48.189:36719)] Successfully connected to peer
07:44:03.811 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.8.48.190:43767(127.8.48.190:43767)] Successfully connected to peer
07:44:03.814 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.189:36719(127.8.48.189:36719)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:44:03.814 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.8.48.190:43767(127.8.48.190:43767)] cleaning up while in state NEGOTIATING due to: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:44:03.814 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.189:36719: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:44:03.814 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.8.48.190:43767: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
07:44:03.814 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.189:36719(127.8.48.189:36719)] handling channelInactive
07:44:03.814 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767; will retry
07:44:03.815 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:44:03.815665 9094 negotiation.cc:337] Failed RPC negotiation. Trace:
07:44:03.815 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.812252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52518 (local address 127.8.48.189:36719)
07:44:03.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.812814 (+ 562us) server_negotiation.cc:206] Beginning negotiation
07:44:03.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.812843 (+ 29us) server_negotiation.cc:399] Waiting for connection header
07:44:03.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.812890 (+ 47us) server_negotiation.cc:407] Connection header received
07:44:03.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.813030 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:44:03.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.813068 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:44:03.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.813282 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:44:03.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.813460 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
07:44:03.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.814722 (+ 1262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52518: BlockingRecv error: recv got EOF from 127.0.0.1:52518 (error 108)
07:44:03.816 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":111}
07:44:03.816 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:1887) Cannot continue with RPC because of: Timed out: cannot complete before timeout: KuduRpc(method=ListTabletServers, tablet=null, attempt=25, TimeoutTracker(timeout=30000, elapsed=29919), Traces: [0ms] refreshing cache from master, [1ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [2ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [4ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [105ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [111ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Configuration error: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [113ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Configuration error: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [114ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but this client is not authenticated (missing or expired TGT),org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [121ms] refreshing cache from master, [122ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [123ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [123ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [126ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [126ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [126ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [127ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [141ms] refreshing cache from master, [141ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [142ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [142ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [147ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [150ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [152ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [153ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [161ms] refreshing cache from master, [161ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [162ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [162ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [172ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [173ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [174ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [175ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [201ms] refreshing cache from master, [202ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [202ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [203ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [206ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [207ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [208ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [209ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [221ms] refreshing cache from master, [221ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [222ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [222ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [228ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [233ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [239ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [239ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [261ms] refreshing cache from master, [261ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [262ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [262ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [274ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [275ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [278ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [278ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [281ms] refreshing cache from master, [281ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [282ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [282ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [288ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [289ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [290ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [292ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [441ms] refreshing cache from master, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [447ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [449ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [450ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [451ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [661ms] refreshing cache from master, [661ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [662ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [662ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [669ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [673ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [681ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [682ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1221ms] refreshing cache from master, [1221ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [1222ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [1222ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [1228ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [1255ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1262ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1262ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2401ms] refreshing cache from master, [2401ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [2402ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [2402ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [2408ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [2420ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2423ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2423ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2841ms] refreshing cache from master, [2841ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [2842ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [2842ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [2846ms] too many traces: truncated at 100 traces, deferred=Deferred@816280786(state=PENDING, result=null, callback=(continuation of Deferred@222163164 after retry RPC after error@1998396020) -> (continuation of Deferred@1169863056 after retry RPC after error@1311788049) -> (continuation of Deferred@753528347 after retry RPC after error@867737416) -> (continuation of Deferred@597593758 after retry RPC after error@1862405688) -> (continuation of Deferred@217623797 after retry RPC after error@2077044989) -> (continuation of Deferred@745581412 after retry RPC after error@1400640284) -> (continuation of Deferred@438993538 after retry RPC after error@141689826) -> (continuation of Deferred@569422339 after retry RPC after error@312975826) -> (continuation of Deferred@1549213590 after retry RPC after error@1143106285) -> (continuation of Deferred@1625729216 after retry RPC after error@1340091455) -> (continuation of Deferred@732897786 after retry RPC after error@2089632608) -> (continuation of Deferred@704140441 after retry RPC after error@1731658829) -> (continuation of Deferred@893737376 after retry RPC after error@591764011) -> (continuation of Deferred@1223843398 after retry RPC after error@389561977) -> (continuation of Deferred@137358695 after retry RPC after error@129280651) -> (continuation of Deferred@1575933013 after retry RPC after error@200536679) -> (continuation of Deferred@2084594347 after retry RPC after error@1585606490) -> (continuation of Deferred@1969570979 after retry RPC after error@1636950178) -> (continuation of Deferred@507220737 after retry RPC after error@1393481507) -> (continuation of Deferred@202494397 after retry RPC after error@1659610221) -> (continuation of Deferred@545947818 after retry RPC after error@93912151) -> (continuation of Deferred@117852512 after retry RPC after error@946953505) -> (continuation of Deferred@1284113749 after retry RPC after error@246559447) -> (continuation of Deferred@994719528 after retry RPC after error@157347125), errback=(continuation of Deferred@222163164 after retry RPC after error@1998396020) -> (continuation of Deferred@1169863056 after retry RPC after error@1311788049) -> (continuation of Deferred@753528347 after retry RPC after error@867737416) -> (continuation of Deferred@597593758 after retry RPC after error@1862405688) -> (continuation of Deferred@217623797 after retry RPC after error@2077044989) -> (continuation of Deferred@745581412 after retry RPC after error@1400640284) -> (continuation of Deferred@438993538 after retry RPC after error@141689826) -> (continuation of Deferred@569422339 after retry RPC after error@312975826) -> (continuation of Deferred@1549213590 after retry RPC after error@1143106285) -> (continuation of Deferred@1625729216 after retry RPC after error@1340091455) -> (continuation of Deferred@732897786 after retry RPC after error@2089632608) -> (continuation of Deferred@704140441 after retry RPC after error@1731658829) -> (continuation of Deferred@893737376 after retry RPC after error@591764011) -> (continuation of Deferred@1223843398 after retry RPC after error@389561977) -> (continuation of Deferred@137358695 after retry RPC after error@129280651) -> (continuation of Deferred@1575933013 after retry RPC after error@200536679) -> (continuation of Deferred@2084594347 after retry RPC after error@1585606490) -> (continuation of Deferred@1969570979 after retry RPC after error@1636950178) -> (continuation of Deferred@507220737 after retry RPC after error@1393481507) -> (continuation of Deferred@202494397 after retry RPC after error@1659610221) -> (continuation of Deferred@545947818 after retry RPC after error@93912151) -> (continuation of Deferred@117852512 after retry RPC after error@946953505) -> (continuation of Deferred@1284113749 after retry RPC after error@246559447) -> (continuation of Deferred@994719528 after retry RPC after error@157347125)))
07:44:03.817 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.8.48.190:43767(127.8.48.190:43767)] handling channelInactive
07:44:03.818 [DEBUG - Time-limited test] (Connection.java:575) [peer master-127.8.48.189:36719(127.8.48.189:36719)] disconnecting while in state TERMINATED
07:44:03.818 [DEBUG - Time-limited test] (Connection.java:575) [peer master-127.8.48.188:34721(127.8.48.188:34721)] disconnecting while in state READY
07:44:03.818 [DEBUG - Time-limited test] (Connection.java:575) [peer master-127.8.48.190:43767(127.8.48.190:43767)] disconnecting while in state TERMINATED
07:44:03.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260620 07:44:03.818674 9095 negotiation.cc:337] Failed RPC negotiation. Trace:
07:44:03.818 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.812512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52634 (local address 127.8.48.190:43767)
07:44:03.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.813014 (+ 502us) server_negotiation.cc:206] Beginning negotiation
07:44:03.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.813040 (+ 26us) server_negotiation.cc:399] Waiting for connection header
07:44:03.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.813084 (+ 44us) server_negotiation.cc:407] Connection header received
07:44:03.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.813207 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
07:44:03.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.813234 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
07:44:03.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.813445 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
07:44:03.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.813608 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
07:44:03.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0620 07:44:03.817829 (+ 4221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52634: BlockingRecv error: recv got EOF from 127.0.0.1:52634 (error 108)
07:44:03.819 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":113}
07:44:03.819 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.8.48.188:34721(127.8.48.188:34721)] handling channelInactive
07:44:03.820 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.8.48.188:34721(127.8.48.188:34721)] cleaning up while in state READY due to: connection closed
07:44:03.820 [DEBUG - kudu-nio-0] (AsyncKuduClient.java:2704) Releasing all remaining resources
07:44:03.821 [DEBUG - main] (AsyncKuduClient.java:2704) Releasing all remaining resources
07:44:03.824 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:44:03.823545 8386 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu with pid 8889
07:44:03.851 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:44:03.850210 8386 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu with pid 8722
07:44:03.881 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:44:03.880198 8386 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu with pid 8796
07:44:05.060 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-06-20T07:44:05Z chronyd exiting
07:44:06.104 [DEBUG - main] (AsyncKuduClient.java:2704) Releasing all remaining resources
07:44:06.106 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:44:06.105973 7663 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu with pid 7946
07:44:06.136 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:44:06.136165 7663 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu with pid 8098
07:44:06.165 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:44:06.165145 7663 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu with pid 8245
07:44:06.192 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:44:06.192075 7663 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu with pid 7675
07:44:06.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:44:06.224159 7663 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu with pid 7745
07:44:06.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260620 07:44:06.255502 7663 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNAFIDr/build/tsan/bin/kudu with pid 7820
07:44:07.398 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-06-20T07:44:07Z chronyd exiting
07:44:08.442 [ERROR - main] (RetryRule.java:206) org.apache.kudu.client.TestSecurity.testExternallyProvidedSubjectExpires: failed attempt 1
org.apache.kudu.client.NonRecoverableException: cannot complete before timeout: KuduRpc(method=ListTabletServers, tablet=null, attempt=25, TimeoutTracker(timeout=30000, elapsed=29919), Traces: [0ms] refreshing cache from master, [1ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [2ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [4ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [105ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [111ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Configuration error: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [113ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Configuration error: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [114ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but this client is not authenticated (missing or expired TGT),org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [121ms] refreshing cache from master, [122ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [123ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [123ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [126ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [126ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [126ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [127ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [141ms] refreshing cache from master, [141ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [142ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [142ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [147ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [150ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [152ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [153ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [161ms] refreshing cache from master, [161ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [162ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [162ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [172ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [173ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [174ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [175ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [201ms] refreshing cache from master, [202ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [202ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [203ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [206ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [207ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [208ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [209ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [221ms] refreshing cache from master, [221ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [222ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [222ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [228ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [233ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [239ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [239ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [261ms] refreshing cache from master, [261ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [262ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [262ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [274ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [275ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [278ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [278ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [281ms] refreshing cache from master, [281ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [282ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [282ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [288ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [289ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [290ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [292ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [441ms] refreshing cache from master, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [447ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [449ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [450ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [451ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [661ms] refreshing cache from master, [661ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [662ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [662ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [669ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [673ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [681ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [682ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1221ms] refreshing cache from master, [1221ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [1222ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [1222ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [1228ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [1255ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1262ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1262ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2401ms] refreshing cache from master, [2401ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [2402ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [2402ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [2408ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [2420ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2423ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2423ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2841ms] refreshing cache from master, [2841ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [2842ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [2842ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [2846ms] too many traces: truncated at 100 traces, deferred=Deferred@816280786(state=PENDING, result=null, callback=(continuation of Deferred@222163164 after retry RPC after error@1998396020) -> (continuation of Deferred@1169863056 after retry RPC after error@1311788049) -> (continuation of Deferred@753528347 after retry RPC after error@867737416) -> (continuation of Deferred@597593758 after retry RPC after error@1862405688) -> (continuation of Deferred@217623797 after retry RPC after error@2077044989) -> (continuation of Deferred@745581412 after retry RPC after error@1400640284) -> (continuation of Deferred@438993538 after retry RPC after error@141689826) -> (continuation of Deferred@569422339 after retry RPC after error@312975826) -> (continuation of Deferred@1549213590 after retry RPC after error@1143106285) -> (continuation of Deferred@1625729216 after retry RPC after error@1340091455) -> (continuation of Deferred@732897786 after retry RPC after error@2089632608) -> (continuation of Deferred@704140441 after retry RPC after error@1731658829) -> (continuation of Deferred@893737376 after retry RPC after error@591764011) -> (continuation of Deferred@1223843398 after retry RPC after error@389561977) -> (continuation of Deferred@137358695 after retry RPC after error@129280651) -> (continuation of Deferred@1575933013 after retry RPC after error@200536679) -> (continuation of Deferred@2084594347 after retry RPC after error@1585606490) -> (continuation of Deferred@1969570979 after retry RPC after error@1636950178) -> (continuation of Deferred@507220737 after retry RPC after error@1393481507) -> (continuation of Deferred@202494397 after retry RPC after error@1659610221) -> (continuation of Deferred@545947818 after retry RPC after error@93912151) -> (continuation of Deferred@117852512 after retry RPC after error@946953505) -> (continuation of Deferred@1284113749 after retry RPC after error@246559447) -> (continuation of Deferred@994719528 after retry RPC after error@157347125), errback=(continuation of Deferred@222163164 after retry RPC after error@1998396020) -> (continuation of Deferred@1169863056 after retry RPC after error@1311788049) -> (continuation of Deferred@753528347 after retry RPC after error@867737416) -> (continuation of Deferred@597593758 after retry RPC after error@1862405688) -> (continuation of Deferred@217623797 after retry RPC after error@2077044989) -> (continuation of Deferred@745581412 after retry RPC after error@1400640284) -> (continuation of Deferred@438993538 after retry RPC after error@141689826) -> (continuation of Deferred@569422339 after retry RPC after error@312975826) -> (continuation of Deferred@1549213590 after retry RPC after error@1143106285) -> (continuation of Deferred@1625729216 after retry RPC after error@1340091455) -> (continuation of Deferred@732897786 after retry RPC after error@2089632608) -> (continuation of Deferred@704140441 after retry RPC after error@1731658829) -> (continuation of Deferred@893737376 after retry RPC after error@591764011) -> (continuation of Deferred@1223843398 after retry RPC after error@389561977) -> (continuation of Deferred@137358695 after retry RPC after error@129280651) -> (continuation of Deferred@1575933013 after retry RPC after error@200536679) -> (continuation of Deferred@2084594347 after retry RPC after error@1585606490) -> (continuation of Deferred@1969570979 after retry RPC after error@1636950178) -> (continuation of Deferred@507220737 after retry RPC after error@1393481507) -> (continuation of Deferred@202494397 after retry RPC after error@1659610221) -> (continuation of Deferred@545947818 after retry RPC after error@93912151) -> (continuation of Deferred@117852512 after retry RPC after error@946953505) -> (continuation of Deferred@1284113749 after retry RPC after error@246559447) -> (continuation of Deferred@994719528 after retry RPC after error@157347125)))
at org.apache.kudu.client.KuduException.transformException(KuduException.java:110) ~[main/:?]
at org.apache.kudu.client.KuduClient.joinAndHandleException(KuduClient.java:564) ~[main/:?]
at org.apache.kudu.client.KuduClient.listTabletServers(KuduClient.java:263) ~[main/:?]
at org.apache.kudu.client.TestSecurity$3.get(TestSecurity.java:386) ~[test/:?]
at org.apache.kudu.test.junit.AssertHelpers.assertEventuallyTrue(AssertHelpers.java:40) ~[kudu-test-utils-1.18.0-SNAPSHOT-unshaded.jar:1.18.0-SNAPSHOT]
at org.apache.kudu.client.TestSecurity.assertEventualAuthenticationFailure(TestSecurity.java:378) ~[test/:?]
at org.apache.kudu.client.TestSecurity.testExternallyProvidedSubjectExpires(TestSecurity.java:412) ~[test/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_362]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_362]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_362]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_362]
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) ~[junit-4.13.2.jar:4.13.2]
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) ~[junit-4.13.2.jar:4.13.2]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_362]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_362]
Suppressed: org.apache.kudu.client.KuduException$OriginalException: Original asynchronous stack trace
at org.apache.kudu.client.AsyncKuduClient.tooManyAttemptsOrTimeout(AsyncKuduClient.java:1889) ~[main/:?]
at org.apache.kudu.client.AsyncKuduClient.delayedSendRpcToTablet(AsyncKuduClient.java:2337) ~[main/:?]
at org.apache.kudu.client.AsyncKuduClient.access$1500(AsyncKuduClient.java:299) ~[main/:?]
at org.apache.kudu.client.AsyncKuduClient$RetryRpcErrback.call(AsyncKuduClient.java:1520) ~[main/:?]
at org.apache.kudu.client.AsyncKuduClient$RetryRpcErrback.call(AsyncKuduClient.java:1510) ~[main/:?]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002) ~[async-1.4.1.jar:?]
at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:313) ~[main/:?]
at org.apache.kudu.client.ConnectToCluster.access$000(ConnectToCluster.java:51) ~[main/:?]
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:387) ~[main/:?]
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:376) ~[main/:?]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002) ~[async-1.4.1.jar:?]
at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:277) ~[main/:?]
at org.apache.kudu.client.KuduRpc.errback(KuduRpc.java:331) ~[main/:?]
at org.apache.kudu.client.RpcProxy.responseReceived(RpcProxy.java:256) ~[main/:?]
at org.apache.kudu.client.RpcProxy.access$000(RpcProxy.java:64) ~[main/:?]
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:158) ~[main/:?]
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:154) ~[main/:?]
at org.apache.kudu.client.Connection.cleanup(Connection.java:737) ~[main/:?]
at org.apache.kudu.client.Connection.exceptionCaught(Connection.java:455) ~[main/:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:447) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.94.Final.jar:4.1.94.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.94.Final.jar:4.1.94.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_362]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_362]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_362]
Caused by: org.apache.kudu.client.NoLeaderFoundException: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:311) ~[main/:?]
at org.apache.kudu.client.ConnectToCluster.access$000(ConnectToCluster.java:51) ~[main/:?]
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:387) ~[main/:?]
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:376) ~[main/:?]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) ~[async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002) ~[async-1.4.1.jar:?]
at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:277) ~[main/:?]
at org.apache.kudu.client.KuduRpc.errback(KuduRpc.java:331) ~[main/:?]
at org.apache.kudu.client.RpcProxy.responseReceived(RpcProxy.java:256) ~[main/:?]
at org.apache.kudu.client.RpcProxy.access$000(RpcProxy.java:64) ~[main/:?]
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:158) ~[main/:?]
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:154) ~[main/:?]
at org.apache.kudu.client.Connection.cleanup(Connection.java:737) ~[main/:?]
at org.apache.kudu.client.Connection.exceptionCaught(Connection.java:455) ~[main/:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:447) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.94.Final.jar:4.1.94.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.94.Final.jar:4.1.94.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_362]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_362]
... 1 more
Caused by: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
at org.apache.kudu.client.Negotiator.chooseAndInitializeSaslMech(Negotiator.java:565) ~[main/:?]
at org.apache.kudu.client.Negotiator.handleNegotiateResponse(Negotiator.java:433) ~[main/:?]
at org.apache.kudu.client.Negotiator.channelRead0(Negotiator.java:367) ~[main/:?]
at org.apache.kudu.client.Negotiator.channelRead0(Negotiator.java:94) ~[main/:?]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[netty-handler-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[netty-codec-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[netty-transport-4.1.94.Final.jar:4.1.94.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.94.Final.jar:4.1.94.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.94.Final.jar:4.1.94.Final]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_362]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_362]
... 1 more
org.apache.kudu.client.NonRecoverableException: cannot complete before timeout: KuduRpc(method=ListTabletServers, tablet=null, attempt=25, TimeoutTracker(timeout=30000, elapsed=29919), Traces: [0ms] refreshing cache from master, [1ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [2ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [4ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [105ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [111ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Configuration error: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [113ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Configuration error: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [114ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but this client is not authenticated (missing or expired TGT),org.apache.kudu.client.NonRecoverableException: Server requires Kerberos, but this client is not authenticated (missing or expired TGT), [121ms] refreshing cache from master, [122ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [123ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [123ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [126ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [126ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [126ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [127ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [141ms] refreshing cache from master, [141ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [142ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [142ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [147ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [150ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [152ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [153ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [161ms] refreshing cache from master, [161ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [162ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [162ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [172ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [173ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [174ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [175ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [201ms] refreshing cache from master, [202ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [202ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [203ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [206ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [207ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [208ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [209ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [221ms] refreshing cache from master, [221ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [222ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [222ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [228ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [233ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [239ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [239ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [261ms] refreshing cache from master, [261ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [262ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [262ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [274ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [275ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [278ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [278ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [281ms] refreshing cache from master, [281ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [282ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [282ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [288ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [289ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [290ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [292ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [441ms] refreshing cache from master, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [442ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [447ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [449ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [450ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [451ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [661ms] refreshing cache from master, [661ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [662ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [662ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [669ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [673ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [681ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [682ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1221ms] refreshing cache from master, [1221ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [1222ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [1222ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [1228ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [1255ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1262ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [1262ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2401ms] refreshing cache from master, [2401ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [2402ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [2402ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [2408ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.188:34721: OK, [2420ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.190:43767: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2423ms] Sub RPC ConnectToMaster: received response from server master-127.8.48.189:36719: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2423ms] delaying RPC due to: Service unavailable: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, [2841ms] refreshing cache from master, [2841ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.189:36719, [2842ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.188:34721, [2842ms] Sub RPC ConnectToMaster: sending RPC to server master-127.8.48.190:43767, [2846ms] too many traces: truncated at 100 traces, deferred=Deferred@816280786(state=PENDING, result=null, callback=(continuation of Deferred@222163164 after retry RPC after error@1998396020) -> (continuation of Deferred@1169863056 after retry RPC after error@1311788049) -> (continuation of Deferred@753528347 after retry RPC after error@867737416) -> (continuation of Deferred@597593758 after retry RPC after error@1862405688) -> (continuation of Deferred@217623797 after retry RPC after error@2077044989) -> (continuation of Deferred@745581412 after retry RPC after error@1400640284) -> (continuation of Deferred@438993538 after retry RPC after error@141689826) -> (continuation of Deferred@569422339 after retry RPC after error@312975826) -> (continuation of Deferred@1549213590 after retry RPC after error@1143106285) -> (continuation of Deferred@1625729216 after retry RPC after error@1340091455) -> (continuation of Deferred@732897786 after retry RPC after error@2089632608) -> (continuation of Deferred@704140441 after retry RPC after error@1731658829) -> (continuation of Deferred@893737376 after retry RPC after error@591764011) -> (continuation of Deferred@1223843398 after retry RPC after error@389561977) -> (continuation of Deferred@137358695 after retry RPC after error@129280651) -> (continuation of Deferred@1575933013 after retry RPC after error@200536679) -> (continuation of Deferred@2084594347 after retry RPC after error@1585606490) -> (continuation of Deferred@1969570979 after retry RPC after error@1636950178) -> (continuation of Deferred@507220737 after retry RPC after error@1393481507) -> (continuation of Deferred@202494397 after retry RPC after error@1659610221) -> (continuation of Deferred@545947818 after retry RPC after error@93912151) -> (continuation of Deferred@117852512 after retry RPC after error@946953505) -> (continuation of Deferred@1284113749 after retry RPC after error@246559447) -> (continuation of Deferred@994719528 after retry RPC after error@157347125), errback=(continuation of Deferred@222163164 after retry RPC after error@1998396020) -> (continuation of Deferred@1169863056 after retry RPC after error@1311788049) -> (continuation of Deferred@753528347 after retry RPC after error@867737416) -> (continuation of Deferred@597593758 after retry RPC after error@1862405688) -> (continuation of Deferred@217623797 after retry RPC after error@2077044989) -> (continuation of Deferred@745581412 after retry RPC after error@1400640284) -> (continuation of Deferred@438993538 after retry RPC after error@141689826) -> (continuation of Deferred@569422339 after retry RPC after error@312975826) -> (continuation of Deferred@1549213590 after retry RPC after error@1143106285) -> (continuation of Deferred@1625729216 after retry RPC after error@1340091455) -> (continuation of Deferred@732897786 after retry RPC after error@2089632608) -> (continuation of Deferred@704140441 after retry RPC after error@1731658829) -> (continuation of Deferred@893737376 after retry RPC after error@591764011) -> (continuation of Deferred@1223843398 after retry RPC after error@389561977) -> (continuation of Deferred@137358695 after retry RPC after error@129280651) -> (continuation of Deferred@1575933013 after retry RPC after error@200536679) -> (continuation of Deferred@2084594347 after retry RPC after error@1585606490) -> (continuation of Deferred@1969570979 after retry RPC after error@1636950178) -> (continuation of Deferred@507220737 after retry RPC after error@1393481507) -> (continuation of Deferred@202494397 after retry RPC after error@1659610221) -> (continuation of Deferred@545947818 after retry RPC after error@93912151) -> (continuation of Deferred@117852512 after retry RPC after error@946953505) -> (continuation of Deferred@1284113749 after retry RPC after error@246559447) -> (continuation of Deferred@994719528 after retry RPC after error@157347125)))
at org.apache.kudu.client.KuduException.transformException(KuduException.java:110)
at org.apache.kudu.client.KuduClient.joinAndHandleException(KuduClient.java:564)
at org.apache.kudu.client.KuduClient.listTabletServers(KuduClient.java:263)
at org.apache.kudu.client.TestSecurity$3.get(TestSecurity.java:386)
at org.apache.kudu.test.junit.AssertHelpers.assertEventuallyTrue(AssertHelpers.java:40)
at org.apache.kudu.client.TestSecurity.assertEventualAuthenticationFailure(TestSecurity.java:378)
at org.apache.kudu.client.TestSecurity.testExternallyProvidedSubjectExpires(TestSecurity.java:412)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:750)
Suppressed: org.apache.kudu.client.KuduException$OriginalException: Original asynchronous stack trace
at org.apache.kudu.client.AsyncKuduClient.tooManyAttemptsOrTimeout(AsyncKuduClient.java:1889)
at org.apache.kudu.client.AsyncKuduClient.delayedSendRpcToTablet(AsyncKuduClient.java:2337)
at org.apache.kudu.client.AsyncKuduClient.access$1500(AsyncKuduClient.java:299)
at org.apache.kudu.client.AsyncKuduClient$RetryRpcErrback.call(AsyncKuduClient.java:1520)
at org.apache.kudu.client.AsyncKuduClient$RetryRpcErrback.call(AsyncKuduClient.java:1510)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002)
at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:313)
at org.apache.kudu.client.ConnectToCluster.access$000(ConnectToCluster.java:51)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:387)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:376)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002)
at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:277)
at org.apache.kudu.client.KuduRpc.errback(KuduRpc.java:331)
at org.apache.kudu.client.RpcProxy.responseReceived(RpcProxy.java:256)
at org.apache.kudu.client.RpcProxy.access$000(RpcProxy.java:64)
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:158)
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:154)
at org.apache.kudu.client.Connection.cleanup(Connection.java:737)
at org.apache.kudu.client.Connection.exceptionCaught(Connection.java:455)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317)
at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:447)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.kudu.client.NoLeaderFoundException: Master config (127.8.48.189:36719,127.8.48.188:34721,127.8.48.190:43767) has no leader. Exceptions received: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available,org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:311)
at org.apache.kudu.client.ConnectToCluster.access$000(ConnectToCluster.java:51)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:387)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:376)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002)
at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:277)
at org.apache.kudu.client.KuduRpc.errback(KuduRpc.java:331)
at org.apache.kudu.client.RpcProxy.responseReceived(RpcProxy.java:256)
at org.apache.kudu.client.RpcProxy.access$000(RpcProxy.java:64)
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:158)
at org.apache.kudu.client.RpcProxy$1.call(RpcProxy.java:154)
at org.apache.kudu.client.Connection.cleanup(Connection.java:737)
at org.apache.kudu.client.Connection.exceptionCaught(Connection.java:455)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317)
at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:447)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.kudu.client.NonRecoverableException: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available
at org.apache.kudu.client.Negotiator.chooseAndInitializeSaslMech(Negotiator.java:565)
at org.apache.kudu.client.Negotiator.handleNegotiateResponse(Negotiator.java:433)
at org.apache.kudu.client.Negotiator.channelRead0(Negotiator.java:367)
at org.apache.kudu.client.Negotiator.channelRead0(Negotiator.java:94)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
... 30 more