01:59:59.296 [INFO - main] (KuduTestHarness.java:155) Creating a new MiniKuduCluster...
01:59:59.296 [INFO - main] (TempDirUtils.java:71) Using the temp directory defined by TEST_TMPDIR: /tmp/dist-test-taskW8An68/test-tmp
01:59:59.296 [INFO - main] (KuduBinaryLocator.java:83) Using Kudu binary directory specified by system property 'kuduBinDir': ../../../build/asan/bin
01:59:59.297 [INFO - main] (MiniKuduCluster.java:217) Starting process: [/tmp/dist-test-taskW8An68/java/build/dist-test/../../../build/asan/bin/kudu, test, mini_cluster, --serialization=pb]
01:59:59.299 [DEBUG - main] (MiniKuduCluster.java:185) Request: create_cluster {
num_masters: 1
num_tservers: 0
enable_kerberos: true
cluster_root: "/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727"
extra_master_flags: "--webserver_require_spnego=false"
extra_master_flags: "--enable-jwt-token-auth"
extra_master_flags: "--rpc-trace-negotiation"
extra_master_flags: "--enable-jwt-token-auth"
extra_master_flags: "--rpc-trace-negotiation"
extra_master_flags: "--rpc-authentication=optional"
extra_master_flags: "--rpc-trace-negotiation"
extra_master_flags: "--user-acl=*"
extra_master_flags: "--enable-jwt-token-auth"
extra_master_flags: "--rpc-authentication=required"
extra_master_flags: "--rpc-negotiation-timeout-ms=1500"
extra_master_flags: "--rpc-trace-negotiation"
extra_master_flags: "--enable-jwt-token-auth"
extra_master_flags: "--rpc-negotiation-timeout-ms=1500"
extra_master_flags: "--rpc-trace-negotiation"
extra_tserver_flags: "--webserver_require_spnego=false"
hms_mode: NONE
mini_kdc_options {
}
principal: "kudu"
mini_oidc_options {
}
}
01:59:59.641 [DEBUG - main] (MiniKuduCluster.java:195) Response:
01:59:59.642 [DEBUG - main] (MiniKuduCluster.java:185) Request: start_cluster {
}
01:59:59.645 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 01:59:59.645200 13669 mini_oidc.cc:150] Starting JWKS server
01:59:59.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 01:59:59.647789 13669 webserver.cc:533] Webserver started at http://127.0.0.1:37321/ using document root <none> and password file <none>
01:59:59.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 01:59:59.648093 13669 mini_oidc.cc:186] Starting OIDC Discovery server
01:59:59.648 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 01:59:59.648689 13669 webserver.cc:533] Webserver started at http://127.0.0.1:46745/ using document root <none> and password file <none>
01:59:59.693 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Loading random data
01:59:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Initializing database '/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/principal' for realm 'KRBTEST.COM',
01:59:59.694 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master key name 'K/M@KRBTEST.COM'
01:59:59.707 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 17 01:59:59 dist-test-slave-g46l krb5kdc[13679](info): setting up network...
01:59:59.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
01:59:59.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 17 01:59:59 dist-test-slave-g46l krb5kdc[13679](info): set up 2 sockets
01:59:59.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 17 01:59:59 dist-test-slave-g46l krb5kdc[13679](info): commencing operation
01:59:59.708 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) krb5kdc: starting...
02:00:00.023 [DEBUG - pool-18-thread-1] (KuduRpc.java:260) Handling a callback on RPC KuduRpc(method=GetTableSchema, tablet=null, attempt=1, TimeoutTracker(timeout=1000, elapsed=1022), Traces: [0ms] refreshing cache from master, [0ms] Sub RPC ConnectToMaster: sending RPC to server master-127.13.55.254:36483, [11ms] Sub RPC ConnectToMaster: received response from server master-127.13.55.254:36483: Not authorized: server requires authentication, but client does not have Kerberos credentials (tgt). Authentication tokens were not used because no token is available, deferred=null) with no deferred attached!
02:00:01.701 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:01.700853 13669 mini_kdc.cc:121] Time spent starting KDC: real 2.044s user 0.003s sys 0.012s
02:00:01.714 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
02:00:01.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
02:00:01.727 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "test-admin@KRBTEST.COM" created.
02:00:01.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
02:00:01.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
02:00:01.753 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "test-user@KRBTEST.COM" created.
02:00:01.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
02:00:01.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
02:00:01.779 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "joe-interloper@KRBTEST.COM" created.
02:00:01.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
02:00:01.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/test-user.keytab.
02:00:01.798 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/test-user.keytab.
02:00:01.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 17 02:00:01 dist-test-slave-g46l krb5kdc[13679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781661601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
02:00:01.820 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Password for test-admin@KRBTEST.COM:
02:00:01.835 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-06-17T02:00:01Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
02:00:01.836 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-06-17T02:00:01Z Disabled control of system clock
02:00:01.858 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for kudu/127.13.89.126@KRBTEST.COM; defaulting to no policy
02:00:01.860 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
02:00:01.860 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "kudu/127.13.89.126@KRBTEST.COM" created.
02:00:01.882 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
02:00:01.882 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.13.89.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/kudu.keytab.
02:00:01.882 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.13.89.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/kudu.keytab.
02:00:01.895 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for HTTP/127.13.89.126@KRBTEST.COM; defaulting to no policy
02:00:01.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
02:00:01.898 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "HTTP/127.13.89.126@KRBTEST.COM" created.
02:00:01.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
02:00:01.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal HTTP/127.13.89.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/kudu.keytab.
02:00:01.918 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal HTTP/127.13.89.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/kudu.keytab.
02:00:01.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:01.920548 13669 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskW8An68/build/asan/bin/kudu
02:00:01.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-taskW8An68/build/asan/bin/kudu
02:00:01.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/wal
02:00:01.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/data
02:00:01.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
02:00:01.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
02:00:01.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
02:00:01.920 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/logs
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/data/info.pb
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.13.89.126:34997
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.13.89.126
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/kudu.keytab
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.13.89.126
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_require_spnego=true
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.13.89.84:45951
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.13.89.126:34997
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:37321/jwks.json
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_require_spnego=false
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable-jwt-token-auth
02:00:01.921 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-trace-negotiation
02:00:01.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable-jwt-token-auth
02:00:01.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-trace-negotiation
02:00:01.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-authentication=optional
02:00:01.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-trace-negotiation
02:00:01.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user-acl=*
02:00:01.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable-jwt-token-auth
02:00:01.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-authentication=required
02:00:01.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-negotiation-timeout-ms=1500
02:00:01.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-trace-negotiation
02:00:01.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable-jwt-token-auth
02:00:01.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-negotiation-timeout-ms=1500
02:00:01.922 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-trace-negotiation with env {KRB5CCNAME=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
02:00:02.270 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.269560 13695 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
02:00:02.270 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.269968 13695 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
02:00:02.270 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.270103 13695 flags.cc:432] Enabled unsafe flag: --never_fsync=true
02:00:02.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.279160 13695 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
02:00:02.279 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.279328 13695 flags.cc:432] Enabled experimental flag: --rpc_trace_negotiation=true
02:00:02.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.279417 13695 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
02:00:02.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.279487 13695 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
02:00:02.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.279569 13695 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
02:00:02.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.279629 13695 flags.cc:432] Enabled experimental flag: --enable_jwt_token_auth=true
02:00:02.280 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.279687 13695 flags.cc:432] Enabled experimental flag: --jwks_url=http://localhost:37321/jwks.json
02:00:02.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.291482 13695 master_runner.cc:387] Master server non-default flags:
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.13.89.84:45951
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/data
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/wal
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.13.89.126:34997
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_trace_negotiation=true
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/kudu.keytab
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.13.89.126
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.13.89.126:34997
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:37321/jwks.json
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_negotiation_timeout_ms=1500
02:00:02.292 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/data/info.pb
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.13.89.126
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/logs
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 23b10557f461bc2be3d2c282748d8de96f9c46ec
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 17 Jun 2026 01:43:24 UTC on bdcb31816ec0
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 12430
02:00:02.293 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ASAN enabled
02:00:02.294 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.293879 13695 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
02:00:02.295 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.295362 13695 file_cache.cc:492] Constructed file cache file cache with capacity 419430
02:00:02.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.304482 13701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:00:02.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.304499 13700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:00:02.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.304430 13703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
02:00:02.305 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.305735 13695 server_base.cc:1061] running on GCE node
02:00:02.306 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.306551 13695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
02:00:02.308 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.308068 13695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
02:00:02.309 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.309336 13695 hybrid_clock.cc:648] HybridClock initialized: now 1781661602309231 us; error 78 us; skew 500 ppm
02:00:02.309 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.309731 13695 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
02:00:02.312 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 17 02:00:02 dist-test-slave-g46l krb5kdc[13679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781661602, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.89.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
02:00:02.313 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.313151 13695 init.cc:377] Logged in from keytab as kudu/127.13.89.126@KRBTEST.COM (short username kudu)
02:00:02.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.314638 13695 webserver.cc:533] Webserver started at http://127.13.89.126:33981/ using document root <none> and password file <none>
02:00:02.315 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.315412 13695 fs_manager.cc:362] Metadata directory not provided
02:00:02.315 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.315505 13695 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
02:00:02.315 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.315840 13695 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
02:00:02.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.318622 13695 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/data/instance:
02:00:02.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "0a65f41e40014ed1877bc33c4482a2be"
02:00:02.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-17 02:00:02 on dist-test-slave-g46l"
02:00:02.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.319480 13695 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/wal/instance:
02:00:02.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "0a65f41e40014ed1877bc33c4482a2be"
02:00:02.319 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-17 02:00:02 on dist-test-slave-g46l"
02:00:02.325 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.325223 13695 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
02:00:02.328 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.328706 13710 log_block_manager.cc:4031] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.330150 13695 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.330294 13695 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/data,/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/wal
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "0a65f41e40014ed1877bc33c4482a2be"
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-06-17 02:00:02 on dist-test-slave-g46l"
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.330477 13695 fs_report.cc:389] FS layout report
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/wal
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/wal
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/data/data
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
02:00:02.330 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
02:00:02.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
02:00:02.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
02:00:02.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
02:00:02.331 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
02:00:02.352 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.352574 13695 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
02:00:02.357 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.356997 13695 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
02:00:02.357 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.357227 13695 kserver.cc:163] Server-wide thread pool size limit: 3276
02:00:02.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.378476 13695 rpc_server.cc:307] RPC server started. Bound to: 127.13.89.126:34997
02:00:02.378 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.378520 13761 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.89.126:34997 every 8 connection(s)
02:00:02.380 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.380100 13695 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/data/info.pb
02:00:02.385 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.385004 13762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
02:00:02.390 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.389762 13669 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskW8An68/build/asan/bin/kudu as pid 13695
02:00:02.390 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.390020 13669 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/master-0/wal/instance
02:00:02.395 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.395339 13762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be: Bootstrap starting.
02:00:02.399 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.399223 13762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be: Neither blocks nor log segments found. Creating new log.
02:00:02.400 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.400766 13762 log.cc:826] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be: Log is configured to *not* fsync() on all Append() calls
02:00:02.404 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.403815 13762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be: No bootstrap required, opened a new log
02:00:02.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.410933 13762 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a65f41e40014ed1877bc33c4482a2be" member_type: VOTER last_known_addr { host: "127.13.89.126" port: 34997 } }
02:00:02.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.411311 13762 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
02:00:02.411 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.411383 13762 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a65f41e40014ed1877bc33c4482a2be, State: Initialized, Role: FOLLOWER
02:00:02.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.412147 13762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a65f41e40014ed1877bc33c4482a2be" member_type: VOTER last_known_addr { host: "127.13.89.126" port: 34997 } }
02:00:02.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.412349 13762 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
02:00:02.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.412451 13762 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
02:00:02.412 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.412577 13762 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [term 0 FOLLOWER]: Advancing to term 1
02:00:02.413 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Jun 17 02:00:02 dist-test-slave-g46l krb5kdc[13679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781661601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.89.126@KRBTEST.COM
02:00:02.415 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.415215 13762 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a65f41e40014ed1877bc33c4482a2be" member_type: VOTER last_known_addr { host: "127.13.89.126" port: 34997 } }
02:00:02.416 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.415910 13762 leader_election.cc:304] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a65f41e40014ed1877bc33c4482a2be; no voters:
02:00:02.417 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.416961 13762 leader_election.cc:290] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [CANDIDATE]: Term 1 election: Requested vote from peers
02:00:02.417 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.417434 13767 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [term 1 FOLLOWER]: Leader election won for term 1
02:00:02.420 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.420373 13767 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [term 1 LEADER]: Becoming Leader. State: Replica: 0a65f41e40014ed1877bc33c4482a2be, State: Running, Role: LEADER
02:00:02.421 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.421259 13767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a65f41e40014ed1877bc33c4482a2be" member_type: VOTER last_known_addr { host: "127.13.89.126" port: 34997 } }
02:00:02.421 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.421540 13762 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [sys.catalog]: configured and running, proceeding with master startup.
02:00:02.425 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.424376 13768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a65f41e40014ed1877bc33c4482a2be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a65f41e40014ed1877bc33c4482a2be" member_type: VOTER last_known_addr { host: "127.13.89.126" port: 34997 } } }
02:00:02.425 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.424685 13768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [sys.catalog]: This master's current role is: LEADER
02:00:02.428 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.428243 13769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a65f41e40014ed1877bc33c4482a2be. Latest consensus state: current_term: 1 leader_uuid: "0a65f41e40014ed1877bc33c4482a2be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a65f41e40014ed1877bc33c4482a2be" member_type: VOTER last_known_addr { host: "127.13.89.126" port: 34997 } } }
02:00:02.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.428966 13769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be [sys.catalog]: This master's current role is: LEADER
02:00:02.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.429224 13764 negotiation.cc:338] RPC negotiation tracing enabled. Trace:
02:00:02.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.394821 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41940 (local address 127.13.89.126:34997)
02:00:02.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.395836 (+ 1015us) server_negotiation.cc:207] Beginning negotiation
02:00:02.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.395858 (+ 22us) server_negotiation.cc:400] Waiting for connection header
02:00:02.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.397309 (+ 1451us) server_negotiation.cc:408] Connection header received
02:00:02.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.400839 (+ 3530us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
02:00:02.429 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.400870 (+ 31us) server_negotiation.cc:462] Received NEGOTIATE request from client
02:00:02.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.401502 (+ 632us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
02:00:02.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.402003 (+ 501us) server_negotiation.cc:227] Negotiated authn=SASL
02:00:02.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.405237 (+ 3234us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.407409 (+ 2172us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
02:00:02.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.408882 (+ 1473us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.409527 (+ 645us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
02:00:02.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.415087 (+ 5560us) server_negotiation.cc:366] Received SASL_INITIATE NegotiatePB request
02:00:02.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.415160 (+ 73us) server_negotiation.cc:882] Received SASL_INITIATE request from client
02:00:02.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.415177 (+ 17us) server_negotiation.cc:893] Client requested to use mechanism: GSSAPI
02:00:02.430 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.415232 (+ 55us) server_negotiation.cc:923] calling sasl_server_start()
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.420037 (+ 4805us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.421285 (+ 1248us) server_negotiation.cc:366] Received SASL_RESPONSE NegotiatePB request
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.421293 (+ 8us) server_negotiation.cc:957] Received SASL_RESPONSE request from client
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.421302 (+ 9us) server_negotiation.cc:968] calling sasl_server_step()
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.421452 (+ 150us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.422041 (+ 589us) server_negotiation.cc:366] Received SASL_RESPONSE NegotiatePB request
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.422049 (+ 8us) server_negotiation.cc:957] Received SASL_RESPONSE request from client
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.422057 (+ 8us) server_negotiation.cc:968] calling sasl_server_step()
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.422800 (+ 743us) server_negotiation.cc:378] Sending SASL_SUCCESS NegotiatePB response
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.426190 (+ 3390us) server_negotiation.cc:1036] Waiting for connection context
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.426588 (+ 398us) server_negotiation.cc:300] Negotiation successful
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.427214 (+ 626us) negotiation.cc:326] Negotiation complete: OK
02:00:02.431 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":493,"threads_started":1}
02:00:02.433 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.433276 13772 catalog_manager.cc:1510] Loading table and tablet metadata into memory...
02:00:02.439 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.439551 13772 catalog_manager.cc:1519] Initializing Kudu cluster ID...
02:00:02.440 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.439700 13669 external_mini_cluster.cc:955] 0 TS(s) registered with all masters
02:00:02.442 [DEBUG - main] (MiniKuduCluster.java:195) Response:
02:00:02.442 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_kdc_env_vars {
}
02:00:02.444 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_kdc_env_vars {
env_vars {
key: "KRB5CCNAME"
value: "/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/krb5cc"
}
env_vars {
key: "KRB5_CONFIG"
value: "/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/krb5.conf"
}
env_vars {
key: "KRB5_KDC_PROFILE"
value: "/tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/kdc.conf"
}
env_vars {
key: "KUDU_ENABLE_KRB5_REALM_FIX"
value: "yes"
}
}
02:00:02.444 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_masters {
}
02:00:02.446 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_masters {
masters {
id {
type: MASTER
index: 0
}
bound_rpc_address {
host: "127.13.89.126"
port: 34997
}
bound_http_address {
host: "127.13.89.126"
port: 33981
}
}
}
02:00:02.446 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_tservers {
}
02:00:02.447 [DEBUG - main] (MiniKuduCluster.java:195) Response:
02:00:02.447 [INFO - main] (KuduTestHarness.java:157) Creating a new Kudu client...
02:00:02.449 [DEBUG - main] (SecurityUtil.java:105) Using ticketCache: /tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/krb5cc
02:00:02.452 [DEBUG - main] (SecurityUtil.java:119) Logged in as subject: test-admin@KRBTEST.COM
02:00:02.459 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.458824 13772 catalog_manager.cc:1382] Generated new cluster ID: 7123f2ff5449457a924c938074409de8
02:00:02.459 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.458992 13772 catalog_manager.cc:1530] Initializing Kudu internal certificate authority...
02:00:02.477 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.476814 13772 catalog_manager.cc:1405] Generated new certificate authority record
02:00:02.479 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.478699 13772 catalog_manager.cc:1539] Loading token signing keys...
02:00:02.480 [DEBUG - main] (SecurityUtil.java:105) Using ticketCache: /tmp/dist-test-taskW8An68/test-tmp/mini-kudu-cluster12090428608283549727/krb5kdc/krb5cc
02:00:02.482 [DEBUG - main] (SecurityUtil.java:119) Logged in as subject: test-admin@KRBTEST.COM
02:00:02.483 [DEBUG - main] (MiniKuduCluster.java:185) Request: create_jwt {
account_id: "account-id"
subject: "kudu"
is_valid_key: false
}
02:00:02.489 [DEBUG - main] (MiniKuduCluster.java:195) Response: create_jwt {
jwt: "eyJhbGciOiJSUzI1NiIsImtpZCI6InB1YmxpYzo5YjlkMGI0Ny1iOWVkLTRiYTYtOTE4MC01MmZjNWIxNjFhM2EiLCJ0eXAiOiJKV1QifQ.eyJleHAiOjE3ODE2NjUyMDIsImlzcyI6ImF1dGgwXC9hY2NvdW50LWlkIiwibmJmIjoxNzgxNjYxNjAyLCJzdWIiOiJrdWR1In0.efoglpMCs4jw6OABbXScZ4NeJm7gQ6glOpq0TRrRhS_kY_-npu41TDAlvx7TY3prQG2qDF5jYr7wpUtqSkgAezrphaxlOWRqrVdCt5Tgn-sRUBQH5_5MrsPuWaDiwcR5XvKeQ_Po_Bk9CbEBBS2ePoyPAxYRNsDFAuhJIVzYs_fLwZ6cvN0lO9C_N2PuVXzhzBI_E6F25duZIrhZlMA1Bp_cYkC2b65j2vrN1qh4R8ikmxa9M4Ppw-OCgyqYbyH1DIjnDQhFuqQNppo1aIx7nOX3D9wPCJJDJMAZbMag8N_dnzHT6bqiE13QmURvQWIAOX9HXXjNm93uf43jqI44DA"
}
02:00:02.490 [DEBUG - main] (SecurityContext.java:317) Importing authentication credentials with no authn token, with one JWT, 0 cert(s), and realUser=<none>
02:00:02.502 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.501858 13772 catalog_manager.cc:6069] T 00000000000000000000000000000000 P 0a65f41e40014ed1877bc33c4482a2be: Generated new TSK 0
02:00:02.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.503695 13772 catalog_manager.cc:1549] Initializing in-progress tserver states...
02:00:02.504 [DEBUG - main] (Connection.java:748) [peer master-127.13.89.126:34997(127.13.89.126:34997)] connecting to peer
02:00:02.507 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.13.89.126:34997(127.13.89.126:34997)] Successfully connected to peer
02:00:02.551 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.551656 13764 negotiation.cc:338] RPC negotiation tracing enabled. Trace:
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.507054 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41944 (local address 127.13.89.126:34997)
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.507224 (+ 170us) server_negotiation.cc:207] Beginning negotiation
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.507229 (+ 5us) server_negotiation.cc:400] Waiting for connection header
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.511724 (+ 4495us) server_negotiation.cc:408] Connection header received
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.512821 (+ 1097us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.512826 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.512940 (+ 114us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.513082 (+ 142us) server_negotiation.cc:227] Negotiated authn=JWT
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.522527 (+ 9445us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.524192 (+ 1665us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.537058 (+ 12866us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.537739 (+ 681us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.539588 (+ 1849us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.549665 (+ 10077us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.551267 (+ 1602us) negotiation.cc:326] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41944: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.552 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":58,"thread_start_us":425,"threads_started":1}
02:00:02.552 [DEBUG - kudu-nio-0] (Negotiator.java:352) peer /127.13.89.126:34997 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.553 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.553153 13764 negotiation.cc:343] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41944: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.556 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.13.89.126:34997(127.13.89.126:34997)] handling channelInactive
02:00:02.556 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.13.89.126:34997(127.13.89.126:34997)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
02:00:02.556 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.13.89.126:34997: connection closed
02:00:02.557 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.13.89.126:34997; will retry
02:00:02.565 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.13.89.126:34997(127.13.89.126:34997)] connecting to peer
02:00:02.567 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.13.89.126:34997(127.13.89.126:34997)] Successfully connected to peer
02:00:02.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.596506 13764 negotiation.cc:338] RPC negotiation tracing enabled. Trace:
02:00:02.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.567816 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41948 (local address 127.13.89.126:34997)
02:00:02.596 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.568072 (+ 256us) server_negotiation.cc:207] Beginning negotiation
02:00:02.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.568079 (+ 7us) server_negotiation.cc:400] Waiting for connection header
02:00:02.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.571820 (+ 3741us) server_negotiation.cc:408] Connection header received
02:00:02.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.572679 (+ 859us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
02:00:02.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.572687 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
02:00:02.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.572800 (+ 113us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
02:00:02.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.572927 (+ 127us) server_negotiation.cc:227] Negotiated authn=JWT
02:00:02.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.581620 (+ 8693us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.583207 (+ 1587us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
02:00:02.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.593780 (+ 10573us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.594483 (+ 703us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
02:00:02.597 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.594698 (+ 215us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
02:00:02.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.595790 (+ 1092us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.596169 (+ 379us) negotiation.cc:326] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41948: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":89}
02:00:02.598 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.597383 13764 negotiation.cc:343] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41948: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.597 [DEBUG - kudu-nio-1] (Negotiator.java:352) peer /127.13.89.126:34997 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.599 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.13.89.126:34997(127.13.89.126:34997)] handling channelInactive
02:00:02.600 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.13.89.126:34997(127.13.89.126:34997)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
02:00:02.600 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.13.89.126:34997: connection closed
02:00:02.600 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.13.89.126:34997; will retry
02:00:02.604 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.13.89.126:34997(127.13.89.126:34997)] connecting to peer
02:00:02.607 [DEBUG - kudu-nio-2] (Connection.java:757) [peer master-127.13.89.126:34997(127.13.89.126:34997)] Successfully connected to peer
02:00:02.636 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.636620 13764 negotiation.cc:338] RPC negotiation tracing enabled. Trace:
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.606977 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41952 (local address 127.13.89.126:34997)
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.607180 (+ 203us) server_negotiation.cc:207] Beginning negotiation
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.607187 (+ 7us) server_negotiation.cc:400] Waiting for connection header
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.611903 (+ 4716us) server_negotiation.cc:408] Connection header received
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.612863 (+ 960us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.612871 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.613004 (+ 133us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.613085 (+ 81us) server_negotiation.cc:227] Negotiated authn=JWT
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.621163 (+ 8078us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.622806 (+ 1643us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.632757 (+ 9951us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.633393 (+ 636us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.634762 (+ 1369us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.635887 (+ 1125us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.636315 (+ 428us) negotiation.cc:326] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41952: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":77}
02:00:02.637 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.637637 13764 negotiation.cc:343] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41952: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.637 [DEBUG - kudu-nio-2] (Negotiator.java:352) peer /127.13.89.126:34997 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.639 [DEBUG - kudu-nio-2] (Connection.java:239) [peer master-127.13.89.126:34997(127.13.89.126:34997)] handling channelInactive
02:00:02.639 [DEBUG - kudu-nio-2] (Connection.java:709) [peer master-127.13.89.126:34997(127.13.89.126:34997)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
02:00:02.640 [INFO - kudu-nio-2] (ConnectToCluster.java:385) Unable to connect to master 127.13.89.126:34997: connection closed
02:00:02.640 [WARN - kudu-nio-2] (ConnectToCluster.java:305) Unable to find the leader master 127.13.89.126:34997; will retry
02:00:02.645 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.13.89.126:34997(127.13.89.126:34997)] connecting to peer
02:00:02.648 [DEBUG - kudu-nio-3] (Connection.java:757) [peer master-127.13.89.126:34997(127.13.89.126:34997)] Successfully connected to peer
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.680574 13764 negotiation.cc:338] RPC negotiation tracing enabled. Trace:
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.648611 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41958 (local address 127.13.89.126:34997)
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.648830 (+ 219us) server_negotiation.cc:207] Beginning negotiation
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.648837 (+ 7us) server_negotiation.cc:400] Waiting for connection header
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.653200 (+ 4363us) server_negotiation.cc:408] Connection header received
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.654242 (+ 1042us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.654251 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.654370 (+ 119us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.654467 (+ 97us) server_negotiation.cc:227] Negotiated authn=JWT
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.665423 (+ 10956us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.666971 (+ 1548us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.676813 (+ 9842us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.677492 (+ 679us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.678811 (+ 1319us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
02:00:02.681 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.679961 (+ 1150us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.680301 (+ 340us) negotiation.cc:326] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41958: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":94}
02:00:02.682 [DEBUG - kudu-nio-3] (Negotiator.java:352) peer /127.13.89.126:34997 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.682 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.680712 13764 negotiation.cc:343] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41958: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.683 [DEBUG - kudu-nio-3] (Connection.java:239) [peer master-127.13.89.126:34997(127.13.89.126:34997)] handling channelInactive
02:00:02.684 [DEBUG - kudu-nio-3] (Connection.java:709) [peer master-127.13.89.126:34997(127.13.89.126:34997)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
02:00:02.684 [INFO - kudu-nio-3] (ConnectToCluster.java:385) Unable to connect to master 127.13.89.126:34997: connection closed
02:00:02.684 [WARN - kudu-nio-3] (ConnectToCluster.java:305) Unable to find the leader master 127.13.89.126:34997; will retry
02:00:02.704 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.13.89.126:34997(127.13.89.126:34997)] connecting to peer
02:00:02.707 [DEBUG - kudu-nio-4] (Connection.java:757) [peer master-127.13.89.126:34997(127.13.89.126:34997)] Successfully connected to peer
02:00:02.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.738564 13764 negotiation.cc:338] RPC negotiation tracing enabled. Trace:
02:00:02.738 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.707391 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41970 (local address 127.13.89.126:34997)
02:00:02.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.707620 (+ 229us) server_negotiation.cc:207] Beginning negotiation
02:00:02.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.707627 (+ 7us) server_negotiation.cc:400] Waiting for connection header
02:00:02.739 [DEBUG - kudu-nio-4] (Negotiator.java:352) peer /127.13.89.126:34997 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.712493 (+ 4866us) server_negotiation.cc:408] Connection header received
02:00:02.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.713334 (+ 841us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
02:00:02.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.713340 (+ 6us) server_negotiation.cc:462] Received NEGOTIATE request from client
02:00:02.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.713424 (+ 84us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
02:00:02.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.713537 (+ 113us) server_negotiation.cc:227] Negotiated authn=JWT
02:00:02.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.724183 (+ 10646us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.725790 (+ 1607us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
02:00:02.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.735627 (+ 9837us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.736352 (+ 725us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
02:00:02.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.736552 (+ 200us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
02:00:02.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.737735 (+ 1183us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.738194 (+ 459us) negotiation.cc:326] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41970: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":89}
02:00:02.740 [DEBUG - kudu-nio-4] (Connection.java:239) [peer master-127.13.89.126:34997(127.13.89.126:34997)] handling channelInactive
02:00:02.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.740527 13764 negotiation.cc:343] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41970: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.741 [DEBUG - kudu-nio-4] (Connection.java:709) [peer master-127.13.89.126:34997(127.13.89.126:34997)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
02:00:02.741 [INFO - kudu-nio-4] (ConnectToCluster.java:385) Unable to connect to master 127.13.89.126:34997: connection closed
02:00:02.741 [WARN - kudu-nio-4] (ConnectToCluster.java:305) Unable to find the leader master 127.13.89.126:34997; will retry
02:00:02.765 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.13.89.126:34997(127.13.89.126:34997)] connecting to peer
02:00:02.767 [DEBUG - kudu-nio-5] (Connection.java:757) [peer master-127.13.89.126:34997(127.13.89.126:34997)] Successfully connected to peer
02:00:02.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.795337 13764 negotiation.cc:338] RPC negotiation tracing enabled. Trace:
02:00:02.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.767592 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41980 (local address 127.13.89.126:34997)
02:00:02.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.767827 (+ 235us) server_negotiation.cc:207] Beginning negotiation
02:00:02.795 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.767834 (+ 7us) server_negotiation.cc:400] Waiting for connection header
02:00:02.795 [DEBUG - kudu-nio-5] (Negotiator.java:352) peer /127.13.89.126:34997 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.771451 (+ 3617us) server_negotiation.cc:408] Connection header received
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.772295 (+ 844us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.772302 (+ 7us) server_negotiation.cc:462] Received NEGOTIATE request from client
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.772423 (+ 121us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.772549 (+ 126us) server_negotiation.cc:227] Negotiated authn=JWT
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.782719 (+ 10170us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.784309 (+ 1590us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.792722 (+ 8413us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.793373 (+ 651us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.793589 (+ 216us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.794649 (+ 1060us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.795041 (+ 392us) negotiation.cc:326] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41980: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.796 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":93}
02:00:02.797 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.796818 13764 negotiation.cc:343] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41980: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.797 [DEBUG - kudu-nio-5] (Connection.java:239) [peer master-127.13.89.126:34997(127.13.89.126:34997)] handling channelInactive
02:00:02.797 [DEBUG - kudu-nio-5] (Connection.java:709) [peer master-127.13.89.126:34997(127.13.89.126:34997)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
02:00:02.797 [INFO - kudu-nio-5] (ConnectToCluster.java:385) Unable to connect to master 127.13.89.126:34997: connection closed
02:00:02.798 [WARN - kudu-nio-5] (ConnectToCluster.java:305) Unable to find the leader master 127.13.89.126:34997; will retry
02:00:02.864 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.13.89.126:34997(127.13.89.126:34997)] connecting to peer
02:00:02.867 [DEBUG - kudu-nio-6] (Connection.java:757) [peer master-127.13.89.126:34997(127.13.89.126:34997)] Successfully connected to peer
02:00:02.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:02.893474 13764 negotiation.cc:338] RPC negotiation tracing enabled. Trace:
02:00:02.893 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.867588 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41982 (local address 127.13.89.126:34997)
02:00:02.894 [DEBUG - kudu-nio-6] (Negotiator.java:352) peer /127.13.89.126:34997 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.867821 (+ 233us) server_negotiation.cc:207] Beginning negotiation
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.867829 (+ 8us) server_negotiation.cc:400] Waiting for connection header
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.871583 (+ 3754us) server_negotiation.cc:408] Connection header received
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.872371 (+ 788us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.872380 (+ 9us) server_negotiation.cc:462] Received NEGOTIATE request from client
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.872492 (+ 112us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.872618 (+ 126us) server_negotiation.cc:227] Negotiated authn=JWT
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.880748 (+ 8130us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.882395 (+ 1647us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.890825 (+ 8430us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.891422 (+ 597us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.891746 (+ 324us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.892903 (+ 1157us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.893215 (+ 312us) negotiation.cc:326] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41982: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.894 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":80}
02:00:02.895 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:02.894912 13764 negotiation.cc:343] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41982: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:02.895 [DEBUG - kudu-nio-6] (Connection.java:239) [peer master-127.13.89.126:34997(127.13.89.126:34997)] handling channelInactive
02:00:02.895 [DEBUG - kudu-nio-6] (Connection.java:709) [peer master-127.13.89.126:34997(127.13.89.126:34997)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
02:00:02.896 [INFO - kudu-nio-6] (ConnectToCluster.java:385) Unable to connect to master 127.13.89.126:34997: connection closed
02:00:02.896 [WARN - kudu-nio-6] (ConnectToCluster.java:305) Unable to find the leader master 127.13.89.126:34997; will retry
02:00:02.984 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.13.89.126:34997(127.13.89.126:34997)] connecting to peer
02:00:02.986 [DEBUG - kudu-nio-7] (Connection.java:757) [peer master-127.13.89.126:34997(127.13.89.126:34997)] Successfully connected to peer
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:03.003844 13764 negotiation.cc:338] RPC negotiation tracing enabled. Trace:
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.986916 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.13.89.126:34997)
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.987101 (+ 185us) server_negotiation.cc:207] Beginning negotiation
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.987106 (+ 5us) server_negotiation.cc:400] Waiting for connection header
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.987758 (+ 652us) server_negotiation.cc:408] Connection header received
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.988454 (+ 696us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.988459 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.988534 (+ 75us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.988629 (+ 95us) server_negotiation.cc:227] Negotiated authn=JWT
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.993596 (+ 4967us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:02.995055 (+ 1459us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.001857 (+ 6802us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:03.004 [DEBUG - kudu-nio-7] (Negotiator.java:352) peer /127.13.89.126:34997 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.002396 (+ 539us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
02:00:03.004 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.002549 (+ 153us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
02:00:03.005 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.003376 (+ 827us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:03.005 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.003607 (+ 231us) negotiation.cc:326] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41988: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:03.005 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":73}
02:00:03.005 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:03.005591 13764 negotiation.cc:343] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41988: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:03.006 [DEBUG - kudu-nio-7] (Connection.java:239) [peer master-127.13.89.126:34997(127.13.89.126:34997)] handling channelInactive
02:00:03.006 [DEBUG - kudu-nio-7] (Connection.java:709) [peer master-127.13.89.126:34997(127.13.89.126:34997)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
02:00:03.007 [INFO - kudu-nio-7] (ConnectToCluster.java:385) Unable to connect to master 127.13.89.126:34997: connection closed
02:00:03.007 [WARN - kudu-nio-7] (ConnectToCluster.java:305) Unable to find the leader master 127.13.89.126:34997; will retry
02:00:03.205 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.13.89.126:34997(127.13.89.126:34997)] connecting to peer
02:00:03.206 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.13.89.126:34997(127.13.89.126:34997)] Successfully connected to peer
02:00:03.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:03.221992 13764 negotiation.cc:338] RPC negotiation tracing enabled. Trace:
02:00:03.222 [DEBUG - kudu-nio-0] (Negotiator.java:352) peer /127.13.89.126:34997 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:03.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.206533 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:41998 (local address 127.13.89.126:34997)
02:00:03.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.206805 (+ 272us) server_negotiation.cc:207] Beginning negotiation
02:00:03.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.206812 (+ 7us) server_negotiation.cc:400] Waiting for connection header
02:00:03.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.206927 (+ 115us) server_negotiation.cc:408] Connection header received
02:00:03.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.207358 (+ 431us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
02:00:03.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.207363 (+ 5us) server_negotiation.cc:462] Received NEGOTIATE request from client
02:00:03.222 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.207438 (+ 75us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
02:00:03.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.207533 (+ 95us) server_negotiation.cc:227] Negotiated authn=JWT
02:00:03.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.211439 (+ 3906us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:03.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.212515 (+ 1076us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
02:00:03.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.219504 (+ 6989us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:03.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.219894 (+ 390us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
02:00:03.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.220372 (+ 478us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
02:00:03.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.221374 (+ 1002us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:03.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.221710 (+ 336us) negotiation.cc:326] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41998: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:03.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":126}
02:00:03.223 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:03.223492 13764 negotiation.cc:343] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41998: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:03.223 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.13.89.126:34997(127.13.89.126:34997)] handling channelInactive
02:00:03.224 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.13.89.126:34997(127.13.89.126:34997)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
02:00:03.224 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.13.89.126:34997: connection closed
02:00:03.224 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.13.89.126:34997; will retry
02:00:03.485 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.13.89.126:34997(127.13.89.126:34997)] connecting to peer
02:00:03.486 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.13.89.126:34997(127.13.89.126:34997)] Successfully connected to peer
02:00:03.502 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:03.502321 13764 negotiation.cc:338] RPC negotiation tracing enabled. Trace:
02:00:03.502 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.486424 (+ 0us) reactor.cc:730] Submitting negotiation task for server connection from 127.0.0.1:42008 (local address 127.13.89.126:34997)
02:00:03.502 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.486652 (+ 228us) server_negotiation.cc:207] Beginning negotiation
02:00:03.502 [DEBUG - kudu-nio-1] (Negotiator.java:352) peer /127.13.89.126:34997 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.486659 (+ 7us) server_negotiation.cc:400] Waiting for connection header
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.486838 (+ 179us) server_negotiation.cc:408] Connection header received
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.487285 (+ 447us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.487293 (+ 8us) server_negotiation.cc:462] Received NEGOTIATE request from client
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.487387 (+ 94us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.487504 (+ 117us) server_negotiation.cc:227] Negotiated authn=JWT
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.491794 (+ 4290us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.493285 (+ 1491us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.499621 (+ 6336us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.500339 (+ 718us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.500526 (+ 187us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.501637 (+ 1111us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0617 02:00:03.502005 (+ 368us) negotiation.cc:326] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42008: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":86}
02:00:03.503 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260617 02:00:03.502492 13764 negotiation.cc:343] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42008: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
02:00:03.504 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.13.89.126:34997(127.13.89.126:34997)] handling channelInactive
02:00:03.504 [INFO - pool-22-thread-1] (ConnectToCluster.java:385) Unable to connect to master 127.13.89.126:34997: cannot complete before timeout: KuduRpc(method=ConnectToMaster, tablet=null, attempt=1, TimeoutTracker(timeout=18, elapsed=18), Traces: [0ms] sending RPC to server master-127.13.89.126:34997, deferred=Deferred@152933393(state=PENDING, result=null, callback=passthrough -> ConnectToMasterCB for 127.13.89.126:34997, errback=org.apache.kudu.client.ConnectToCluster$1@46240bdc -> ConnectToMasterErrCB for 127.13.89.126:34997))
02:00:03.504 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.13.89.126:34997(127.13.89.126:34997)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
02:00:03.504 [DEBUG - kudu-nio-1] (KuduRpc.java:260) Handling a callback on RPC KuduRpc(method=ConnectToMaster, tablet=null, attempt=0, TimeoutTracker(timeout=0, elapsed=0), Traces: [0ms] received response from server master-127.13.89.126:34997: Network error: connection closed, deferred=null) with no deferred attached!
02:00:03.505 [DEBUG - pool-22-thread-1] (AsyncKuduClient.java:1523) Notify RPC KuduRpc(method=GetTableSchema, tablet=null, attempt=10, TimeoutTracker(timeout=1000, elapsed=1002), Traces: [0ms] refreshing cache from master, [1ms] Sub RPC ConnectToMaster: sending RPC to server master-127.13.89.126:34997, [53ms] Sub RPC ConnectToMaster: received response from server master-127.13.89.126:34997: Network error: connection closed, [54ms] delaying RPC due to: Service unavailable: Master config (127.13.89.126:34997) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [61ms] refreshing cache from master, [61ms] Sub RPC ConnectToMaster: sending RPC to server master-127.13.89.126:34997, [97ms] Sub RPC ConnectToMaster: received response from server master-127.13.89.126:34997: Network error: connection closed, [97ms] delaying RPC due to: Service unavailable: Master config (127.13.89.126:34997) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [101ms] refreshing cache from master, [101ms] Sub RPC ConnectToMaster: sending RPC to server master-127.13.89.126:34997, [136ms] Sub RPC ConnectToMaster: received response from server master-127.13.89.126:34997: Network error: connection closed, [137ms] delaying RPC due to: Service unavailable: Master config (127.13.89.126:34997) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [141ms] refreshing cache from master, [142ms] Sub RPC ConnectToMaster: sending RPC to server master-127.13.89.126:34997, [181ms] Sub RPC ConnectToMaster: received response from server master-127.13.89.126:34997: Network error: connection closed, [181ms] delaying RPC due to: Service unavailable: Master config (127.13.89.126:34997) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [201ms] refreshing cache from master, [201ms] Sub RPC ConnectToMaster: sending RPC to server master-127.13.89.126:34997, [238ms] Sub RPC ConnectToMaster: received response from server master-127.13.89.126:34997: Network error: connection closed, [238ms] delaying RPC due to: Service unavailable: Master config (127.13.89.126:34997) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [261ms] refreshing cache from master, [262ms] Sub RPC ConnectToMaster: sending RPC to server master-127.13.89.126:34997, [294ms] Sub RPC ConnectToMaster: received response from server master-127.13.89.126:34997: Network error: connection closed, [295ms] delaying RPC due to: Service unavailable: Master config (127.13.89.126:34997) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [361ms] refreshing cache from master, [361ms] Sub RPC ConnectToMaster: sending RPC to server master-127.13.89.126:34997, [392ms] Sub RPC ConnectToMaster: received response from server master-127.13.89.126:34997: Network error: connection closed, [393ms] delaying RPC due to: Service unavailable: Master config (127.13.89.126:34997) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [481ms] refreshing cache from master, [481ms] Sub RPC ConnectToMaster: sending RPC to server master-127.13.89.126:34997, [504ms] Sub RPC ConnectToMaster: received response from server master-127.13.89.126:34997: Network error: connection closed, [504ms] delaying RPC due to: Service unavailable: Master config (127.13.89.126:34997) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [701ms] refreshing cache from master, [701ms] Sub RPC ConnectToMaster: sending RPC to server master-127.13.89.126:34997, [721ms] Sub RPC ConnectToMaster: received response from server master-127.13.89.126:34997: Network error: connection closed, [721ms] delaying RPC due to: Service unavailable: Master config (127.13.89.126:34997) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [981ms] refreshing cache from master, [982ms] Sub RPC ConnectToMaster: sending RPC to server master-127.13.89.126:34997, deferred=Deferred@750769473(state=PENDING, result=null, callback=(continuation of Deferred@1124978871 after retry RPC after error@643131867) -> (continuation of Deferred@1078590502 after retry RPC after error@302922931) -> (continuation of Deferred@1059839533 after retry RPC after error@851272610) -> (continuation of Deferred@1025114399 after retry RPC after error@1586645419) -> (continuation of Deferred@1068052194 after retry RPC after error@1724967622) -> (continuation of Deferred@187633146 after retry RPC after error@1195168293) -> (continuation of Deferred@823409182 after retry RPC after error@1349742624) -> (continuation of Deferred@599844617 after retry RPC after error@46965934) -> (continuation of Deferred@303606307 after retry RPC after error@1495345712), errback=(continuation of Deferred@1124978871 after retry RPC after error@643131867) -> (continuation of Deferred@1078590502 after retry RPC after error@302922931) -> (continuation of Deferred@1059839533 after retry RPC after error@851272610) -> (continuation of Deferred@1025114399 after retry RPC after error@1586645419) -> (continuation of Deferred@1068052194 after retry RPC after error@1724967622) -> (continuation of Deferred@187633146 after retry RPC after error@1195168293) -> (continuation of Deferred@823409182 after retry RPC after error@1349742624) -> (continuation of Deferred@599844617 after retry RPC after error@46965934) -> (continuation of Deferred@303606307 after retry RPC after error@1495345712))) after lookup exception
org.apache.kudu.client.NonRecoverableException: Couldn't find a valid master in (127.13.89.126:34997). Exceptions received: [org.apache.kudu.client.NonRecoverableException: cannot complete before timeout: KuduRpc(method=ConnectToMaster, tablet=null, attempt=1, TimeoutTracker(timeout=18, elapsed=18), Traces: [0ms] sending RPC to server master-127.13.89.126:34997, deferred=Deferred@152933393(state=PENDING, result=null, callback=passthrough -> ConnectToMasterCB for 127.13.89.126:34997, errback=org.apache.kudu.client.ConnectToCluster$1@46240bdc -> ConnectToMasterErrCB for 127.13.89.126:34997))]
at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:268)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:387)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:376)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002)
at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:277)
at org.apache.kudu.client.KuduRpc.errback(KuduRpc.java:331)
at org.apache.kudu.client.KuduRpc$RpcTimeoutTask.run(KuduRpc.java:466)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:713)
at io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:701)
at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:787)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:501)
at java.base/java.lang.Thread.run(Thread.java:840)
org.apache.kudu.client.NonRecoverableException: Couldn't find a valid master in (127.13.89.126:34997). Exceptions received: [org.apache.kudu.client.NonRecoverableException: cannot complete before timeout: KuduRpc(method=ConnectToMaster, tablet=null, attempt=1, TimeoutTracker(timeout=18, elapsed=18), Traces: [0ms] sending RPC to server master-127.13.89.126:34997, deferred=Deferred@152933393(state=PENDING, result=null, callback=passthrough -> ConnectToMasterCB for 127.13.89.126:34997, errback=org.apache.kudu.client.ConnectToCluster$1@46240bdc -> ConnectToMasterErrCB for 127.13.89.126:34997))]
at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:268)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:387)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:376)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002)
at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:277)
at org.apache.kudu.client.KuduRpc.errback(KuduRpc.java:331)
at org.apache.kudu.client.KuduRpc$RpcTimeoutTask.run(KuduRpc.java:466)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:713)
at io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:701)
at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:787)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:501)
at java.base/java.lang.Thread.run(Thread.java:840)
02:00:03.506 [DEBUG - main] (AsyncKuduClient.java:2704) Releasing all remaining resources
02:00:03.510 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:03.509855 13669 external_mini_cluster.cc:1705] Attempting to check leaks for /tmp/dist-test-taskW8An68/build/asan/bin/kudu pid 13695
02:00:03.511 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:03.511371 13757 generic_service.cc:196] Checking for leaks (request via RPC)
02:00:03.621 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260617 02:00:03.620795 13669 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskW8An68/build/asan/bin/kudu with pid 13695
02:00:03.665 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-06-17T02:00:03Z chronyd exiting
02:00:03.773 [ERROR - main] (RetryRule.java:206) org.apache.kudu.client.TestNegotiation.testInvalidJwt: failed attempt 1
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:55)
at junit.framework.Assert.assertTrue(Assert.java:22)
at junit.framework.Assert.assertTrue(Assert.java:31)
at junit.framework.TestCase.assertTrue(TestCase.java:200)
at org.apache.kudu.client.TestNegotiation.testInvalidJwt(TestNegotiation.java:280)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at org.apache.kudu.test.junit.RetryRule$RetryStatement.doOneAttemptAndReport(RetryRule.java:184)
at org.apache.kudu.test.junit.RetryRule$RetryStatement.evaluate(RetryRule.java:244)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
at org.junit.runner.JUnitCore.main(JUnitCore.java:36)
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:55)
at junit.framework.Assert.assertTrue(Assert.java:22)
at junit.framework.Assert.assertTrue(Assert.java:31)
at junit.framework.TestCase.assertTrue(TestCase.java:200)
at org.apache.kudu.client.TestNegotiation.testInvalidJwt(TestNegotiation.java:280)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at org.apache.kudu.test.junit.RetryRule$RetryStatement.doOneAttemptAndReport(RetryRule.java:184)
at org.apache.kudu.test.junit.RetryRule$RetryStatement.evaluate(RetryRule.java:244)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
at org.junit.runner.JUnitCore.main(JUnitCore.java:36)