09:07:16.105 [INFO - main] (KuduTestHarness.java:155) Creating a new MiniKuduCluster...
09:07:16.105 [INFO - main] (TempDirUtils.java:71) Using the temp directory defined by TEST_TMPDIR: /tmp/dist-test-task5fNXJ8/test-tmp
09:07:16.106 [INFO - main] (KuduBinaryLocator.java:83) Using Kudu binary directory specified by system property 'kuduBinDir': ../../../build/tsan/bin
09:07:16.106 [INFO - main] (MiniKuduCluster.java:217) Starting process: [/tmp/dist-test-task5fNXJ8/java/build/dist-test/../../../build/tsan/bin/kudu, test, mini_cluster, --serialization=pb]
09:07:16.107 [DEBUG - main] (MiniKuduCluster.java:185) Request: create_cluster {
num_masters: 1
num_tservers: 0
enable_kerberos: true
cluster_root: "/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930"
extra_master_flags: "--webserver_require_spnego=false"
extra_master_flags: "--enable-jwt-token-auth"
extra_master_flags: "--rpc-trace-negotiation"
extra_master_flags: "--enable-jwt-token-auth"
extra_master_flags: "--rpc-trace-negotiation"
extra_master_flags: "--rpc-authentication=optional"
extra_master_flags: "--rpc-trace-negotiation"
extra_master_flags: "--user-acl=*"
extra_master_flags: "--enable-jwt-token-auth"
extra_master_flags: "--rpc-authentication=required"
extra_master_flags: "--rpc-negotiation-timeout-ms=1500"
extra_master_flags: "--rpc-trace-negotiation"
extra_master_flags: "--enable-jwt-token-auth"
extra_master_flags: "--rpc-negotiation-timeout-ms=1500"
extra_master_flags: "--rpc-trace-negotiation"
extra_tserver_flags: "--webserver_require_spnego=false"
hms_mode: NONE
mini_kdc_options {
}
principal: "kudu"
mini_oidc_options {
}
}
09:07:16.409 [DEBUG - main] (MiniKuduCluster.java:195) Response:
09:07:16.410 [DEBUG - main] (MiniKuduCluster.java:185) Request: start_cluster {
}
09:07:16.414 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:16.413985 29629 mini_oidc.cc:150] Starting JWKS server
09:07:16.417 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:16.416975 29629 webserver.cc:492] Webserver started at http://127.0.0.1:39579/ using document root <none> and password file <none>
09:07:16.417 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:16.417357 29629 mini_oidc.cc:186] Starting OIDC Discovery server
09:07:16.419 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:16.418901 29629 webserver.cc:492] Webserver started at http://127.0.0.1:33173/ using document root <none> and password file <none>
09:07:16.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Loading random data
09:07:16.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Initializing database '/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/principal' for realm 'KRBTEST.COM',
09:07:16.485 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master key name 'K/M@KRBTEST.COM'
09:07:16.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Feb 04 09:07:16 dist-test-slave-3t09 krb5kdc[29640](info): setting up network...
09:07:16.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
09:07:16.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Feb 04 09:07:16 dist-test-slave-3t09 krb5kdc[29640](info): set up 2 sockets
09:07:16.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Feb 04 09:07:16 dist-test-slave-3t09 krb5kdc[29640](info): commencing operation
09:07:16.497 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) krb5kdc: starting...
09:07:18.266 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.266515 29629 mini_kdc.cc:121] Time spent starting KDC: real 1.814s user 0.008s sys 0.008s
09:07:18.278 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
09:07:18.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
09:07:18.290 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "test-admin@KRBTEST.COM" created.
09:07:18.302 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
09:07:18.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
09:07:18.314 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "test-user@KRBTEST.COM" created.
09:07:18.326 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
09:07:18.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
09:07:18.339 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "joe-interloper@KRBTEST.COM" created.
09:07:18.357 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal slave/admin@KRBTEST.COM with password.
09:07:18.357 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/test-user.keytab.
09:07:18.357 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/test-user.keytab.
09:07:18.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Feb 04 09:07:18 dist-test-slave-3t09 krb5kdc[29640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1770196038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
09:07:18.375 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Password for test-admin@KRBTEST.COM:
09:07:18.388 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-02-04T09:07:18Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
09:07:18.389 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-02-04T09:07:18Z Disabled control of system clock
09:07:18.406 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for kudu/127.28.239.126@KRBTEST.COM; defaulting to no policy
09:07:18.409 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
09:07:18.409 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "kudu/127.28.239.126@KRBTEST.COM" created.
09:07:18.425 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
09:07:18.425 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.28.239.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/kudu.keytab.
09:07:18.425 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal kudu/127.28.239.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/kudu.keytab.
09:07:18.436 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: no policy specified for HTTP/127.28.239.126@KRBTEST.COM; defaulting to no policy
09:07:18.438 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
09:07:18.438 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Principal "HTTP/127.28.239.126@KRBTEST.COM" created.
09:07:18.454 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Authenticating as principal test-admin/admin@KRBTEST.COM with password.
09:07:18.454 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal HTTP/127.28.239.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/kudu.keytab.
09:07:18.454 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Entry for principal HTTP/127.28.239.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/kudu.keytab.
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:18.456843 29629 external_mini_cluster.cc:1366] Running /tmp/dist-test-task5fNXJ8/build/tsan/bin/kudu
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) /tmp/dist-test-task5fNXJ8/build/tsan/bin/kudu
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/wal
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/data
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --block_manager=log
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=localhost
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/logs
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/data/info.pb
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
09:07:18.457 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) master
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) run
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.28.239.126:43219
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.28.239.126
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/kudu.keytab
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.28.239.126
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=test-admin
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user_acl=test-user
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_require_spnego=true
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.28.239.84:38045
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.28.239.126:43219
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:39579/jwks.json
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_require_spnego=false
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable-jwt-token-auth
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-trace-negotiation
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable-jwt-token-auth
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-trace-negotiation
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-authentication=optional
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-trace-negotiation
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --user-acl=*
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable-jwt-token-auth
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-authentication=required
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-negotiation-timeout-ms=1500
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-trace-negotiation
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable-jwt-token-auth
09:07:18.458 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-negotiation-timeout-ms=1500
09:07:18.459 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc-trace-negotiation with env {KRB5CCNAME=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
09:07:18.739 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.739403 29656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
09:07:18.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.739933 29656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
09:07:18.740 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.740625 29656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
09:07:18.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.771437 29656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
09:07:18.771 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.771754 29656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
09:07:18.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.771989 29656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
09:07:18.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.772184 29656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
09:07:18.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.772375 29656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
09:07:18.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.772560 29656 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
09:07:18.772 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.772697 29656 flags.cc:425] Enabled experimental flag: --jwks_url=http://localhost:39579/jwks.json
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:18.808588 29656 master_runner.cc:386] Master server non-default flags:
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_poll_interval_ms=100
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --builtin_ntp_servers=127.28.239.84:38045
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ntp_initial_sync_wait_secs=10
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --time_source=builtin
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_data_dirs=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/data
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --fs_wal_dir=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/wal
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_ca_key_size=768
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --master_addresses=127.28.239.126:43219
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_trace_negotiation=true
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --keytab_file=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/kudu.keytab
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --principal=kudu/127.28.239.126
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --ipki_server_key_size=768
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --openssl_security_level_override=0
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --tsk_num_rsa_bits=512
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_bind_addresses=127.28.239.126:43219
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_reuseport=true
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_server_allow_ephemeral_ports=true
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_jwt_token_auth=true
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --jwks_url=http://localhost:39579/jwks.json
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_authentication=required
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --rpc_negotiation_timeout_ms=1500
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --superuser_acl=<redacted>
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --metrics_log_interval_ms=1000
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_format=pb
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --server_dump_info_path=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/data/info.pb
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_interface=127.28.239.126
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --webserver_port=0
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --never_fsync=true
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --redact=none
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_experimental_flags=true
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --unlock_unsafe_flags=true
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --enable_minidumps=false
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --log_dir=/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/logs
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logbuflevel=-1
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --logtostderr=true
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Master server version:
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) kudu 1.19.0-SNAPSHOT
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) revision 0efbebd487e7d0c7f21ab0d5a91e0fa70046af83
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build type FASTDEBUG
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) built by None at 04 Feb 2026 08:43:20 UTC on 5fd53c4cbb9d
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) build id 10341
09:07:18.809 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) TSAN enabled
09:07:18.810 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:18.809804 29656 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
09:07:18.810 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:18.810719 29656 file_cache.cc:492] Constructed file cache file cache with capacity 419430
09:07:18.821 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.821398 29662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
09:07:19.982 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:18.821842 29663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
09:07:19.983 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:19.982739 29664 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1158 milliseconds
09:07:19.984 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:19.983820 29665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
09:07:19.986 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:19.986512 29656 thread.cc:641] OpenStack (cloud detector) Time spent creating pthread: real 1.165s user 0.460s sys 0.700s
09:07:19.986 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:19.986761 29656 thread.cc:608] OpenStack (cloud detector) Time spent starting thread: real 1.166s user 0.460s sys 0.700s
09:07:19.987 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:19.986940 29656 server_base.cc:1055] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
09:07:19.988 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:19.988016 29656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
09:07:19.990 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:19.990571 29656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
09:07:19.992 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:19.991918 29656 hybrid_clock.cc:648] HybridClock initialized: now 1770196039991872 us; error 61 us; skew 500 ppm
09:07:19.993 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:19.992761 29656 server_base.cc:860] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
09:07:19.998 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Feb 04 09:07:19 dist-test-slave-3t09 krb5kdc[29640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1770196039, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.239.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
09:07:19.999 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:19.999372 29656 init.cc:377] Logged in from keytab as kudu/127.28.239.126@KRBTEST.COM (short username kudu)
09:07:20.005 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.004951 29656 webserver.cc:492] Webserver started at http://127.28.239.126:46813/ using document root <none> and password file <none>
09:07:20.006 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.006074 29656 fs_manager.cc:362] Metadata directory not provided
09:07:20.006 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.006294 29656 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
09:07:20.006 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.006790 29656 server_base.cc:908] This appears to be a new deployment of Kudu; creating new FS layout
09:07:20.011 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.011118 29656 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/data/instance:
09:07:20.011 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "17d0f8d911d84288b8d9b186eaecb61c"
09:07:20.011 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-02-04 09:07:20 on dist-test-slave-3t09"
09:07:20.012 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.012327 29656 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/wal/instance:
09:07:20.012 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "17d0f8d911d84288b8d9b186eaecb61c"
09:07:20.012 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-02-04 09:07:20 on dist-test-slave-3t09"
09:07:20.020 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.020097 29656 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
09:07:20.026 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.026153 29673 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
09:07:20.027 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.027248 29656 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
09:07:20.027 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.027531 29656 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/data,/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/wal
09:07:20.027 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) uuid: "17d0f8d911d84288b8d9b186eaecb61c"
09:07:20.027 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) format_stamp: "Formatted at 2026-02-04 09:07:20 on dist-test-slave-3t09"
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.027830 29656 fs_report.cc:389] FS layout report
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) --------------------
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) wal directory: /tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/wal
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) metadata directory: /tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/wal
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 1 data directories: /tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/data/data
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live blocks: 0
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes: 0
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total live bytes (after alignment): 0
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total number of LBM containers: 0 (0 full)
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for missing blocks
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Did not check for orphaned blocks
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM containers with extra space: 0 (0 repaired)
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total full LBM container extra space in bytes: 0 (0 repaired)
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total incomplete LBM containers: 0 (0 repaired)
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total LBM partial records: 0 (0 repaired)
09:07:20.028 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
09:07:20.088 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.088177 29656 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
09:07:20.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.097992 29656 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
09:07:20.098 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.098531 29656 kserver.cc:163] Server-wide thread pool size limit: 3276
09:07:20.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.164075 29656 rpc_server.cc:307] RPC server started. Bound to: 127.28.239.126:43219
09:07:20.164 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.164160 29724 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.28.239.126:43219 every 8 connection(s)
09:07:20.166 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.166652 29656 server_base.cc:1192] Dumped server information to /tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/data/info.pb
09:07:20.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.170995 29629 external_mini_cluster.cc:1428] Started /tmp/dist-test-task5fNXJ8/build/tsan/bin/kudu as pid 29656
09:07:20.171 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.171540 29629 external_mini_cluster.cc:1442] Reading /tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/master-0/wal/instance
09:07:20.173 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.173091 29725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
09:07:20.194 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.193786 29725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c: Bootstrap starting.
09:07:20.200 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.199712 29725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c: Neither blocks nor log segments found. Creating new log.
09:07:20.201 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.201629 29725 log.cc:826] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c: Log is configured to *not* fsync() on all Append() calls
09:07:20.206 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Feb 04 09:07:20 dist-test-slave-3t09 krb5kdc[29640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1770196038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.239.126@KRBTEST.COM
09:07:20.207 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.206980 29725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c: No bootstrap required, opened a new log
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.219054 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.177268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43698 (local address 127.28.239.126:43219)
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.179962 (+ 2694us) server_negotiation.cc:207] Beginning negotiation
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.180010 (+ 48us) server_negotiation.cc:400] Waiting for connection header
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.181156 (+ 1146us) server_negotiation.cc:408] Connection header received
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.192247 (+ 11091us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.192299 (+ 52us) server_negotiation.cc:462] Received NEGOTIATE request from client
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.192823 (+ 524us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.193277 (+ 454us) server_negotiation.cc:227] Negotiated authn=SASL
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.196946 (+ 3669us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.199168 (+ 2222us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.200688 (+ 1520us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.201339 (+ 651us) server_negotiation.cc:658] Negotiated auth-only TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.207814 (+ 6475us) server_negotiation.cc:366] Received SASL_INITIATE NegotiatePB request
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.207913 (+ 99us) server_negotiation.cc:882] Received SASL_INITIATE request from client
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.207943 (+ 30us) server_negotiation.cc:893] Client requested to use mechanism: GSSAPI
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.208058 (+ 115us) server_negotiation.cc:923] calling sasl_server_start()
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.211966 (+ 3908us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.213555 (+ 1589us) server_negotiation.cc:366] Received SASL_RESPONSE NegotiatePB request
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.213583 (+ 28us) server_negotiation.cc:957] Received SASL_RESPONSE request from client
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.213613 (+ 30us) server_negotiation.cc:968] calling sasl_server_step()
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.213788 (+ 175us) server_negotiation.cc:378] Sending SASL_CHALLENGE NegotiatePB response
09:07:20.219 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.214490 (+ 702us) server_negotiation.cc:366] Received SASL_RESPONSE NegotiatePB request
09:07:20.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.214508 (+ 18us) server_negotiation.cc:957] Received SASL_RESPONSE request from client
09:07:20.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.214524 (+ 16us) server_negotiation.cc:968] calling sasl_server_step()
09:07:20.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.215117 (+ 593us) server_negotiation.cc:378] Sending SASL_SUCCESS NegotiatePB response
09:07:20.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.215432 (+ 315us) server_negotiation.cc:1036] Waiting for connection context
09:07:20.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.216652 (+ 1220us) server_negotiation.cc:300] Negotiation successful
09:07:20.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.217113 (+ 461us) negotiation.cc:327] Negotiation complete: OK
09:07:20.220 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":2010,"thread_start_us":1579,"threads_started":1}
09:07:20.224 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.223860 29629 external_mini_cluster.cc:949] 0 TS(s) registered with all masters
09:07:20.229 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.228688 29725 raft_consensus.cc:359] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d0f8d911d84288b8d9b186eaecb61c" member_type: VOTER last_known_addr { host: "127.28.239.126" port: 43219 } }
09:07:20.229 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.229598 29725 raft_consensus.cc:385] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
09:07:20.230 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.229883 29725 raft_consensus.cc:740] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17d0f8d911d84288b8d9b186eaecb61c, State: Initialized, Role: FOLLOWER
09:07:20.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.230682 29725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d0f8d911d84288b8d9b186eaecb61c" member_type: VOTER last_known_addr { host: "127.28.239.126" port: 43219 } }
09:07:20.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.231335 29725 raft_consensus.cc:399] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
09:07:20.231 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.231652 29725 raft_consensus.cc:493] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
09:07:20.232 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.232018 29725 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [term 0 FOLLOWER]: Advancing to term 1
09:07:20.234 [DEBUG - main] (MiniKuduCluster.java:195) Response:
09:07:20.234 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_kdc_env_vars {
}
09:07:20.235 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_kdc_env_vars {
env_vars {
key: "KRB5CCNAME"
value: "/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/krb5cc"
}
env_vars {
key: "KRB5_CONFIG"
value: "/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/krb5.conf"
}
env_vars {
key: "KRB5_KDC_PROFILE"
value: "/tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/kdc.conf"
}
env_vars {
key: "KUDU_ENABLE_KRB5_REALM_FIX"
value: "yes"
}
}
09:07:20.235 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_masters {
}
09:07:20.237 [DEBUG - main] (MiniKuduCluster.java:195) Response: get_masters {
masters {
id {
type: MASTER
index: 0
}
bound_rpc_address {
host: "127.28.239.126"
port: 43219
}
bound_http_address {
host: "127.28.239.126"
port: 46813
}
}
}
09:07:20.237 [DEBUG - main] (MiniKuduCluster.java:185) Request: get_tservers {
}
09:07:20.237 [DEBUG - main] (MiniKuduCluster.java:195) Response:
09:07:20.237 [INFO - main] (KuduTestHarness.java:157) Creating a new Kudu client...
09:07:20.237 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.237303 29725 raft_consensus.cc:515] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d0f8d911d84288b8d9b186eaecb61c" member_type: VOTER last_known_addr { host: "127.28.239.126" port: 43219 } }
09:07:20.238 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.238152 29725 leader_election.cc:304] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17d0f8d911d84288b8d9b186eaecb61c; no voters:
09:07:20.238 [DEBUG - main] (SecurityUtil.java:105) Using ticketCache: /tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/krb5cc
09:07:20.240 [DEBUG - main] (SecurityUtil.java:119) Logged in as subject: test-admin@KRBTEST.COM
09:07:20.241 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.240166 29725 leader_election.cc:290] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [CANDIDATE]: Term 1 election: Requested vote from peers
09:07:20.241 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.240566 29730 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [term 1 FOLLOWER]: Leader election won for term 1
09:07:20.242 [DEBUG - main] (SecurityUtil.java:105) Using ticketCache: /tmp/dist-test-task5fNXJ8/test-tmp/mini-kudu-cluster5491470583902185930/krb5kdc/krb5cc
09:07:20.243 [DEBUG - main] (SecurityUtil.java:119) Logged in as subject: test-admin@KRBTEST.COM
09:07:20.244 [DEBUG - main] (MiniKuduCluster.java:185) Request: create_jwt {
account_id: "account-id"
subject: "kudu"
is_valid_key: false
}
09:07:20.244 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.244516 29730 raft_consensus.cc:697] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [term 1 LEADER]: Becoming Leader. State: Replica: 17d0f8d911d84288b8d9b186eaecb61c, State: Running, Role: LEADER
09:07:20.246 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.245546 29730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d0f8d911d84288b8d9b186eaecb61c" member_type: VOTER last_known_addr { host: "127.28.239.126" port: 43219 } }
09:07:20.247 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.247170 29725 sys_catalog.cc:565] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [sys.catalog]: configured and running, proceeding with master startup.
09:07:20.252 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.252322 29732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17d0f8d911d84288b8d9b186eaecb61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d0f8d911d84288b8d9b186eaecb61c" member_type: VOTER last_known_addr { host: "127.28.239.126" port: 43219 } } }
09:07:20.253 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.252923 29732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [sys.catalog]: This master's current role is: LEADER
09:07:20.254 [DEBUG - main] (MiniKuduCluster.java:195) Response: create_jwt {
jwt: "eyJhbGciOiJSUzI1NiIsImtpZCI6InB1YmxpYzo5YjlkMGI0Ny1iOWVkLTRiYTYtOTE4MC01MmZjNWIxNjFhM2EiLCJ0eXAiOiJKV1QifQ.eyJleHAiOjE3NzAxOTk2NDAsImlzcyI6ImF1dGgwXC9hY2NvdW50LWlkIiwibmJmIjoxNzcwMTk2MDQwLCJzdWIiOiJrdWR1In0.dfmaZ1Zf5oToECQBtQhofaa47FPxyrw5QROjfGWfnJ-nA9EJI6_yxEMEQS6CFkbSxowuJ9uDK1N9P3i3uLKNZQNy3-t5aK0UhFFoR5LYHqmEbzOAcy_VojPO0Nm6BrDhNHzX6BPZ8v1Pn7-sxejK9BYKoAduVNseFudPrxbCGI8JuyAiTzLoqXlmxYKjv-RmzOhpkHPhddfanNfB7FIQA93T4RWo4BsSpVJ3RAYq7vJvsVE1NlZecoA1jxaJDzJGQFyHTMd9WEUrx_wL4iy4rYLdJoBZLP0VQIAMP0vG2lXxXRE5_Y_RV3aDhTgG1zkzUFHeUoufpmv2rbiy8Cafqw"
}
09:07:20.254 [DEBUG - main] (SecurityContext.java:317) Importing authentication credentials with no authn token, with one JWT, 0 cert(s), and realUser=<none>
09:07:20.255 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.255218 29730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17d0f8d911d84288b8d9b186eaecb61c. Latest consensus state: current_term: 1 leader_uuid: "17d0f8d911d84288b8d9b186eaecb61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17d0f8d911d84288b8d9b186eaecb61c" member_type: VOTER last_known_addr { host: "127.28.239.126" port: 43219 } } }
09:07:20.256 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.255700 29730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c [sys.catalog]: This master's current role is: LEADER
09:07:20.265 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.265209 29737 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
09:07:20.275 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.275559 29737 catalog_manager.cc:1494] Initializing Kudu cluster ID...
09:07:20.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.291002 29737 catalog_manager.cc:1357] Generated new cluster ID: 8bb67be7765d4600a87f6fa0cae5c679
09:07:20.291 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.291244 29737 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
09:07:20.310 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.310179 29737 catalog_manager.cc:1380] Generated new certificate authority record
09:07:20.311 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.311576 29737 catalog_manager.cc:1514] Loading token signing keys...
09:07:20.335 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.334887 29737 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 17d0f8d911d84288b8d9b186eaecb61c: Generated new TSK 0
09:07:20.336 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.335981 29737 catalog_manager.cc:1524] Initializing in-progress tserver states...
09:07:20.504 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ==================
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: ThreadSanitizer: data race (pid=29656)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Read of size 8 at 0x7b1400015c48 by thread T78:
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #0 std::__1::unique_ptr<kudu::security::Cert, std::__1::default_delete<kudu::security::Cert> >::operator bool() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:2502:19 (libmaster.so+0x37ba64)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #1 kudu::master::MasterCertAuthority::IsInitialized() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master_cert_authority.h:70:14 (libmaster.so+0x37ba2d)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #2 kudu::master::MasterPathHandlers::HandleIpkiCaCert(kudu::security::DataFormat, kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master_path_handlers.cc:619:19 (libmaster.so+0x42113e)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #3 kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9::operator()(kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master_path_handlers.cc:916:15 (libmaster.so+0x42f61a)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #4 decltype(std::__1::forward<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9&>(fp)(std::__1::forward<kudu::WebCallbackRegistry::WebRequest const&>(fp0), std::__1::forward<kudu::WebCallbackRegistry::PrerenderedWebResponse*>(fp0))) std::__1::__invoke<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9&, kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*>(kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9&, kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libmaster.so+0x42f5c8)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #5 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9&, kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*>(kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9&, kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libmaster.so+0x42f52c)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #6 std::__1::__function::__alloc_func<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9, std::__1::allocator<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9>, void (kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*)>::operator()(kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libmaster.so+0x42f4c8)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #7 std::__1::__function::__func<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9, std::__1::allocator<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9>, void (kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*)>::operator()(kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libmaster.so+0x42e790)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #8 std::__1::__function::__value_func<void (kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*)>::operator()(kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*&&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libserver_process.so+0x1a5dda)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #9 std::__1::function<void (kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*)>::operator()(kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libserver_process.so+0x19a411)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #10 kudu::Webserver::RunPathHandler(kudu::Webserver::PathHandler const&, sq_connection*, sq_request_info*, kudu::WebCallbackRegistry::PrerenderedWebResponse*, std::__1::unordered_map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/server/webserver.cc:784:5 (libserver_process.so+0x1944ad)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #11 kudu::Webserver::BeginRequestCallback(sq_connection*, sq_request_info*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/server/webserver.cc:715:10 (libserver_process.so+0x1928b7)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #12 kudu::Webserver::BeginRequestCallbackStatic(sq_connection*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/server/webserver.cc:588:20 (libserver_process.so+0x191668)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #13 handle_request /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:3563:27 (libserver_process.so+0x1c8187)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #14 process_new_connection /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:4292:7 (libserver_process.so+0x1c6047)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #15 worker_thread /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:4424:9 (libserver_process.so+0x1c6047)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Previous write of size 8 at 0x7b1400015c48 by thread T77 (mutexes: write M3607):
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #0 std::__1::unique_ptr<kudu::security::Cert, std::__1::default_delete<kudu::security::Cert> >::reset(kudu::security::Cert*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:2515:20 (libmaster.so+0x3a6c22)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #1 std::__1::unique_ptr<kudu::security::Cert, std::__1::default_delete<kudu::security::Cert> >::operator=(std::__1::unique_ptr<kudu::security::Cert, std::__1::default_delete<kudu::security::Cert> >&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:2436:5 (libmaster.so+0x419d38)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #2 kudu::master::MasterCertAuthority::Init(std::__1::unique_ptr<kudu::security::PrivateKey, std::__1::default_delete<kudu::security::PrivateKey> >, std::__1::unique_ptr<kudu::security::Cert, std::__1::default_delete<kudu::security::Cert> >) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master_cert_authority.cc:96:12 (libmaster.so+0x4190cb)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #3 kudu::master::CatalogManager::InitCertAuthorityWith(std::__1::unique_ptr<kudu::security::PrivateKey, std::__1::default_delete<kudu::security::PrivateKey> >, std::__1::unique_ptr<kudu::security::Cert, std::__1::default_delete<kudu::security::Cert> >) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1246:3 (libmaster.so+0x31a036)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #4 kudu::master::CatalogManager::InitCertAuthority() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1223:12 (libmaster.so+0x319a28)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #5 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1507:42 (libmaster.so+0x350e38)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #6 decltype(std::__1::forward<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&>(fp)()) std::__1::__invoke<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&>(kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libmaster.so+0x350df0)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #7 kudu::Status std::__1::__invoke_void_return_wrapper<kudu::Status>::__call<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&>(kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:317:16 (libmaster.so+0x350d80)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #8 std::__1::__function::__alloc_func<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9, std::__1::allocator<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9>, kudu::Status ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libmaster.so+0x350d38)
09:07:20.505 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #9 std::__1::__function::__func<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9, std::__1::allocator<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9>, kudu::Status ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libmaster.so+0x350024)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #10 std::__1::__function::__value_func<kudu::Status ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libmaster.so+0x37c7fb)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #11 std::__1::function<kudu::Status ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libmaster.so+0x37c7a0)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #12 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()(std::__1::function<kudu::Status ()>, kudu::consensus::RaftConsensus const&, long, char const*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1450:24 (libmaster.so+0x31c201)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #13 kudu::master::CatalogManager::PrepareForLeadershipTask() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1507:12 (libmaster.so+0x31b5fb)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #14 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1088:57 (libmaster.so+0x34a511)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #15 decltype(std::__1::forward<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&>(fp)()) std::__1::__invoke<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&>(kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libmaster.so+0x34a4c9)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #16 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&>(kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libmaster.so+0x34a459)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #17 std::__1::__function::__alloc_func<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2, std::__1::allocator<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libmaster.so+0x34a421)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #18 std::__1::__function::__func<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2, std::__1::allocator<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libmaster.so+0x34971d)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #19 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (kudu+0x5aa2b4)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #20 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (kudu+0x5aa0e9)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #21 kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7 (libkudu_util.so+0x467355)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #22 kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48 (libkudu_util.so+0x4696a1)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #23 decltype(std::__1::forward<kudu::ThreadPool::CreateThread()::$_1&>(fp)()) std::__1::__invoke<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkudu_util.so+0x469659)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #24 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libkudu_util.so+0x4695e9)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #25 std::__1::__function::__alloc_func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkudu_util.so+0x4695b1)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #26 std::__1::__function::__func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkudu_util.so+0x4688ad)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #27 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (kudu+0x5aa2b4)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #28 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (kudu+0x5aa0e9)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #29 kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x44a69a)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Location is heap block of size 72 at 0x7b1400015c20 allocated by main thread:
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #0 operator new(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:64 (kudu+0x4f3637)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #1 kudu::master::Master::Init() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master.cc:331:25 (libmaster.so+0x4015c9)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #2 kudu::master::RunMasterServer() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master_runner.cc:395:3 (libmaster.so+0x435b15)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #3 kudu::tools::(anonymous namespace)::MasterRun(kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_action_master.cc:136:10 (kudu+0x5b84f9)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #4 decltype(std::__1::forward<kudu::Status (*&)(kudu::tools::RunnerContext const&)>(fp)(std::__1::forward<kudu::tools::RunnerContext const&>(fp0))) std::__1::__invoke<kudu::Status (*&)(kudu::tools::RunnerContext const&), kudu::tools::RunnerContext const&>(kudu::Status (*&)(kudu::tools::RunnerContext const&), kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (kudu+0x5355d1)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #5 kudu::Status std::__1::__invoke_void_return_wrapper<kudu::Status>::__call<kudu::Status (*&)(kudu::tools::RunnerContext const&), kudu::tools::RunnerContext const&>(kudu::Status (*&)(kudu::tools::RunnerContext const&), kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:317:16 (kudu+0x53553c)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #6 std::__1::__function::__alloc_func<kudu::Status (*)(kudu::tools::RunnerContext const&), std::__1::allocator<kudu::Status (*)(kudu::tools::RunnerContext const&)>, kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (kudu+0x5354ac)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #7 std::__1::__function::__func<kudu::Status (*)(kudu::tools::RunnerContext const&), std::__1::allocator<kudu::Status (*)(kudu::tools::RunnerContext const&)>, kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (kudu+0x534115)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #8 std::__1::__function::__value_func<kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libkudu_tools_util.so+0x14045f)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #9 std::__1::function<kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libkudu_tools_util.so+0x132cc1)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #10 kudu::tools::Action::Run(std::__1::vector<kudu::tools::Mode*, std::__1::allocator<kudu::tools::Mode*> > const&, std::__1::unordered_map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > > const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_action.cc:245:10 (libkudu_tools_util.so+0x130a52)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #11 kudu::tools::DispatchCommand(std::__1::vector<kudu::tools::Mode*, std::__1::allocator<kudu::tools::Mode*> > const&, kudu::tools::Action*, std::__1::deque<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_main.cc:142:15 (kudu+0x659330)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #12 kudu::tools::RunTool(int, char**, bool) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_main.cc:214:16 (kudu+0x659c67)
09:07:20.506 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #13 main /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_main.cc:309:10 (kudu+0x65a1ae)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Mutex M3607 (0x7b5400001a58) created at:
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #0 pthread_rwlock_init /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1317 (kudu+0x4a842b)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #1 kudu::RWMutex::Init(kudu::RWMutex::Priority) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/rw_mutex.cc:78:8 (libkudu_util.so+0x4298d8)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #2 kudu::RWMutex::RWMutex(kudu::RWMutex::Priority) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/rw_mutex.cc:56:3 (libkudu_util.so+0x429b28)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #3 kudu::master::CatalogManager::CatalogManager(kudu::master::Master*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:987:7 (libmaster.so+0x317a0b)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #4 kudu::master::Master::Master(kudu::master::MasterOptions const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master.cc:268:28 (libmaster.so+0x400a0f)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #5 kudu::master::RunMasterServer() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master_runner.cc:394:33 (libmaster.so+0x435ae3)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #6 kudu::tools::(anonymous namespace)::MasterRun(kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_action_master.cc:136:10 (kudu+0x5b84f9)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #7 decltype(std::__1::forward<kudu::Status (*&)(kudu::tools::RunnerContext const&)>(fp)(std::__1::forward<kudu::tools::RunnerContext const&>(fp0))) std::__1::__invoke<kudu::Status (*&)(kudu::tools::RunnerContext const&), kudu::tools::RunnerContext const&>(kudu::Status (*&)(kudu::tools::RunnerContext const&), kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (kudu+0x5355d1)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #8 kudu::Status std::__1::__invoke_void_return_wrapper<kudu::Status>::__call<kudu::Status (*&)(kudu::tools::RunnerContext const&), kudu::tools::RunnerContext const&>(kudu::Status (*&)(kudu::tools::RunnerContext const&), kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:317:16 (kudu+0x53553c)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #9 std::__1::__function::__alloc_func<kudu::Status (*)(kudu::tools::RunnerContext const&), std::__1::allocator<kudu::Status (*)(kudu::tools::RunnerContext const&)>, kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (kudu+0x5354ac)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #10 std::__1::__function::__func<kudu::Status (*)(kudu::tools::RunnerContext const&), std::__1::allocator<kudu::Status (*)(kudu::tools::RunnerContext const&)>, kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (kudu+0x534115)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #11 std::__1::__function::__value_func<kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libkudu_tools_util.so+0x14045f)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #12 std::__1::function<kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libkudu_tools_util.so+0x132cc1)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #13 kudu::tools::Action::Run(std::__1::vector<kudu::tools::Mode*, std::__1::allocator<kudu::tools::Mode*> > const&, std::__1::unordered_map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > > const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_action.cc:245:10 (libkudu_tools_util.so+0x130a52)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #14 kudu::tools::DispatchCommand(std::__1::vector<kudu::tools::Mode*, std::__1::allocator<kudu::tools::Mode*> > const&, kudu::tools::Action*, std::__1::deque<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_main.cc:142:15 (kudu+0x659330)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #15 kudu::tools::RunTool(int, char**, bool) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_main.cc:214:16 (kudu+0x659c67)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #16 main /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_main.cc:309:10 (kudu+0x65a1ae)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Thread T78 'sq_worker' (tid=29738, running) created by thread T13 at:
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #0 pthread_create /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:966 (kudu+0x477c45)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #1 sq_start_thread /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:804:12 (libserver_process.so+0x1c43a0)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #2 try_start_another_worker /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:4449:7 (libserver_process.so+0x1c43a0)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #3 produce_socket /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:4464:5 (libserver_process.so+0x1c43a0)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #4 accept_new_connection /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:4521:5 (libserver_process.so+0x1c43a0)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #5 master_thread /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:4559:11 (libserver_process.so+0x1c43a0)
09:07:20.507 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Thread T77 'leader-initiali' (tid=29737, running) created by thread T72 at:
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #0 pthread_create /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:966 (kudu+0x477c45)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #1 kudu::Thread::StartThread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:643:15 (libkudu_util.so+0x449f38)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #2 kudu::Thread::Create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, scoped_refptr<kudu::Thread>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.h:147:12 (kudu+0x59a529)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #3 kudu::ThreadPool::CreateThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:814:10 (libkudu_util.so+0x46657b)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #4 kudu::ThreadPool::DoSubmit(std::__1::function<void ()>, kudu::ThreadPoolToken*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:599:21 (libkudu_util.so+0x46481f)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #5 kudu::ThreadPool::Submit(std::__1::function<void ()>) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:491:10 (libkudu_util.so+0x466ae2)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #6 kudu::master::CatalogManager::ElectedAsLeaderCb() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1088:33 (libmaster.so+0x319139)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #7 kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1680:40 (libmaster.so+0x355538)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #8 decltype(std::__1::forward<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15&>(fp)()) std::__1::__invoke<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15&>(kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libmaster.so+0x3554f0)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #9 kudu::Status std::__1::__invoke_void_return_wrapper<kudu::Status>::__call<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15&>(kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:317:16 (libmaster.so+0x355480)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #10 std::__1::__function::__alloc_func<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15, std::__1::allocator<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15>, kudu::Status ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libmaster.so+0x355438)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #11 std::__1::__function::__func<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15, std::__1::allocator<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15>, kudu::Status ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libmaster.so+0x354724)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #12 std::__1::__function::__value_func<kudu::Status ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libmaster.so+0x37c7fb)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #13 std::__1::function<kudu::Status ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libmaster.so+0x37c7a0)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #14 kudu::master::SysCatalogTable::SysCatalogStateChanged(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/sys_catalog.cc:461:16 (libmaster.so+0x479759)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #15 kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/sys_catalog.cc:498:15 (libmaster.so+0x481bdc)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #16 decltype(std::__1::forward<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0&>(fp)(std::__1::forward<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(fp0))) std::__1::__invoke<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libmaster.so+0x481b88)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #17 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libmaster.so+0x481b08)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #18 std::__1::__function::__alloc_func<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0, std::__1::allocator<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0>, void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libmaster.so+0x481ac3)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #19 std::__1::__function::__func<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0, std::__1::allocator<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0>, void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libmaster.so+0x480d6c)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #20 std::__1::__function::__value_func<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libtablet.so+0x3ed5b6)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #21 std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libtablet.so+0x3ed548)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #22 kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/raft_consensus.cc:2857:3 (libconsensus.so+0x20421a)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #23 decltype(std::__1::forward<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16&>(fp)()) std::__1::__invoke<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16&>(kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libconsensus.so+0x2041c9)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #24 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16&>(kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libconsensus.so+0x204159)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #25 std::__1::__function::__alloc_func<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16, std::__1::allocator<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libconsensus.so+0x204121)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #26 std::__1::__function::__func<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16, std::__1::allocator<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libconsensus.so+0x2033dd)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #27 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (kudu+0x5aa2b4)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #28 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (kudu+0x5aa0e9)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #29 kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7 (libkudu_util.so+0x467355)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #30 kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48 (libkudu_util.so+0x4696a1)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #31 decltype(std::__1::forward<kudu::ThreadPool::CreateThread()::$_1&>(fp)()) std::__1::__invoke<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkudu_util.so+0x469659)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #32 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libkudu_util.so+0x4695e9)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #33 std::__1::__function::__alloc_func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkudu_util.so+0x4695b1)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #34 std::__1::__function::__func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkudu_util.so+0x4688ad)
09:07:20.508 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #35 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (kudu+0x5aa2b4)
09:07:20.509 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #36 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (kudu+0x5aa0e9)
09:07:20.509 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #37 kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x44a69a)
09:07:20.509 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:20.509 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) SUMMARY: ThreadSanitizer: data race /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:2502:19 in std::__1::unique_ptr<kudu::security::Cert, std::__1::default_delete<kudu::security::Cert> >::operator bool() const
09:07:20.509 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ==================
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ==================
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) WARNING: ThreadSanitizer: data race (pid=29656)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Read of size 8 at 0x7b10000360c0 by thread T78:
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #0 std::__1::unique_ptr<stack_st_X509, std::__1::function<void (stack_st_X509*)> >::get() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:2490:19 (libsecurity.so+0x98e34)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #1 kudu::security::Cert::ToString(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, kudu::security::DataFormat) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/security/cert.cc:98:57 (libsecurity.so+0xa1ffd)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #2 kudu::master::MasterPathHandlers::HandleIpkiCaCert(kudu::security::DataFormat, kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master_path_handlers.cc:626:21 (libmaster.so+0x42116f)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #3 kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9::operator()(kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master_path_handlers.cc:916:15 (libmaster.so+0x42f61a)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #4 decltype(std::__1::forward<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9&>(fp)(std::__1::forward<kudu::WebCallbackRegistry::WebRequest const&>(fp0), std::__1::forward<kudu::WebCallbackRegistry::PrerenderedWebResponse*>(fp0))) std::__1::__invoke<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9&, kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*>(kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9&, kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libmaster.so+0x42f5c8)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #5 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9&, kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*>(kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9&, kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libmaster.so+0x42f52c)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #6 std::__1::__function::__alloc_func<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9, std::__1::allocator<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9>, void (kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*)>::operator()(kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libmaster.so+0x42f4c8)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #7 std::__1::__function::__func<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9, std::__1::allocator<kudu::master::MasterPathHandlers::Register(kudu::Webserver*)::$_9>, void (kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*)>::operator()(kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libmaster.so+0x42e790)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #8 std::__1::__function::__value_func<void (kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*)>::operator()(kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*&&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libserver_process.so+0x1a5dda)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #9 std::__1::function<void (kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*)>::operator()(kudu::WebCallbackRegistry::WebRequest const&, kudu::WebCallbackRegistry::PrerenderedWebResponse*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libserver_process.so+0x19a411)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #10 kudu::Webserver::RunPathHandler(kudu::Webserver::PathHandler const&, sq_connection*, sq_request_info*, kudu::WebCallbackRegistry::PrerenderedWebResponse*, std::__1::unordered_map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/server/webserver.cc:784:5 (libserver_process.so+0x1944ad)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #11 kudu::Webserver::BeginRequestCallback(sq_connection*, sq_request_info*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/server/webserver.cc:715:10 (libserver_process.so+0x1928b7)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #12 kudu::Webserver::BeginRequestCallbackStatic(sq_connection*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/server/webserver.cc:588:20 (libserver_process.so+0x191668)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #13 handle_request /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:3563:27 (libserver_process.so+0x1c8187)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #14 process_new_connection /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:4292:7 (libserver_process.so+0x1c6047)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #15 worker_thread /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:4424:9 (libserver_process.so+0x1c6047)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Previous write of size 8 at 0x7b10000360c0 by thread T77 (mutexes: write M3607):
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #0 std::__1::unique_ptr<stack_st_X509, std::__1::function<void (stack_st_X509*)> >::reset(stack_st_X509*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:2515:20 (libsecurity.so+0xaa282)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #1 std::__1::unique_ptr<stack_st_X509, std::__1::function<void (stack_st_X509*)> >::operator=(std::__1::unique_ptr<stack_st_X509, std::__1::function<void (stack_st_X509*)> >&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:2436:5 (libsecurity.so+0xa8078)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #2 kudu::security::RawDataWrapper<stack_st_X509>::AdoptRawData(stack_st_X509*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/openssl_util.h:262:11 (libsecurity.so+0xa4e94)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #3 kudu::security::Cert::AdoptX509(x509_st*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/security/cert.cc:287:3 (libsecurity.so+0xa322e)
09:07:20.527 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #4 kudu::security::ca::CertSigner::Sign(kudu::security::CertSignRequest const&, kudu::security::Cert*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/security/ca/cert_management.cc:313:8 (libsecurity.so+0x8f5a6)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #5 kudu::security::ca::CertSigner::SelfSignCA(kudu::security::PrivateKey const&, kudu::security::ca::CaCertRequestGenerator::Config, long, kudu::security::Cert*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/security/ca/cert_management.cc:268:8 (libsecurity.so+0x8f3b9)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #6 kudu::master::MasterCertAuthority::Generate(kudu::security::PrivateKey*, kudu::security::Cert*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master_cert_authority.cc:73:10 (libmaster.so+0x418ddf)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #7 kudu::master::CatalogManager::InitCertAuthority() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1217:5 (libmaster.so+0x31998c)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #8 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1507:42 (libmaster.so+0x350e38)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #9 decltype(std::__1::forward<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&>(fp)()) std::__1::__invoke<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&>(kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libmaster.so+0x350df0)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #10 kudu::Status std::__1::__invoke_void_return_wrapper<kudu::Status>::__call<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&>(kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:317:16 (libmaster.so+0x350d80)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #11 std::__1::__function::__alloc_func<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9, std::__1::allocator<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9>, kudu::Status ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libmaster.so+0x350d38)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #12 std::__1::__function::__func<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9, std::__1::allocator<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9>, kudu::Status ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libmaster.so+0x350024)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #13 std::__1::__function::__value_func<kudu::Status ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libmaster.so+0x37c7fb)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #14 std::__1::function<kudu::Status ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libmaster.so+0x37c7a0)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #15 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()(std::__1::function<kudu::Status ()>, kudu::consensus::RaftConsensus const&, long, char const*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1450:24 (libmaster.so+0x31c201)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #16 kudu::master::CatalogManager::PrepareForLeadershipTask() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1507:12 (libmaster.so+0x31b5fb)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #17 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1088:57 (libmaster.so+0x34a511)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #18 decltype(std::__1::forward<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&>(fp)()) std::__1::__invoke<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&>(kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libmaster.so+0x34a4c9)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #19 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&>(kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libmaster.so+0x34a459)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #20 std::__1::__function::__alloc_func<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2, std::__1::allocator<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libmaster.so+0x34a421)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #21 std::__1::__function::__func<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2, std::__1::allocator<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libmaster.so+0x34971d)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #22 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (kudu+0x5aa2b4)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #23 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (kudu+0x5aa0e9)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #24 kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7 (libkudu_util.so+0x467355)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #25 kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48 (libkudu_util.so+0x4696a1)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #26 decltype(std::__1::forward<kudu::ThreadPool::CreateThread()::$_1&>(fp)()) std::__1::__invoke<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkudu_util.so+0x469659)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #27 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libkudu_util.so+0x4695e9)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #28 std::__1::__function::__alloc_func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkudu_util.so+0x4695b1)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #29 std::__1::__function::__func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkudu_util.so+0x4688ad)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #30 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (kudu+0x5aa2b4)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #31 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (kudu+0x5aa0e9)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #32 kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x44a69a)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Location is heap block of size 64 at 0x7b10000360c0 allocated by thread T77:
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #0 operator new(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:64 (kudu+0x4f3637)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #1 kudu::master::CatalogManager::InitCertAuthority() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1214:27 (libmaster.so+0x31994f)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #2 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1507:42 (libmaster.so+0x350e38)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #3 decltype(std::__1::forward<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&>(fp)()) std::__1::__invoke<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&>(kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libmaster.so+0x350df0)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #4 kudu::Status std::__1::__invoke_void_return_wrapper<kudu::Status>::__call<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&>(kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:317:16 (libmaster.so+0x350d80)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #5 std::__1::__function::__alloc_func<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9, std::__1::allocator<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9>, kudu::Status ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libmaster.so+0x350d38)
09:07:20.528 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #6 std::__1::__function::__func<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9, std::__1::allocator<kudu::master::CatalogManager::PrepareForLeadershipTask()::$_9>, kudu::Status ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libmaster.so+0x350024)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #7 std::__1::__function::__value_func<kudu::Status ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libmaster.so+0x37c7fb)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #8 std::__1::function<kudu::Status ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libmaster.so+0x37c7a0)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #9 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()(std::__1::function<kudu::Status ()>, kudu::consensus::RaftConsensus const&, long, char const*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1450:24 (libmaster.so+0x31c201)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #10 kudu::master::CatalogManager::PrepareForLeadershipTask() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1507:12 (libmaster.so+0x31b5fb)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #11 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1088:57 (libmaster.so+0x34a511)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #12 decltype(std::__1::forward<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&>(fp)()) std::__1::__invoke<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&>(kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libmaster.so+0x34a4c9)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #13 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&>(kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libmaster.so+0x34a459)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #14 std::__1::__function::__alloc_func<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2, std::__1::allocator<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libmaster.so+0x34a421)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #15 std::__1::__function::__func<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2, std::__1::allocator<kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libmaster.so+0x34971d)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #16 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (kudu+0x5aa2b4)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #17 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (kudu+0x5aa0e9)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #18 kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7 (libkudu_util.so+0x467355)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #19 kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48 (libkudu_util.so+0x4696a1)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #20 decltype(std::__1::forward<kudu::ThreadPool::CreateThread()::$_1&>(fp)()) std::__1::__invoke<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkudu_util.so+0x469659)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #21 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libkudu_util.so+0x4695e9)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #22 std::__1::__function::__alloc_func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkudu_util.so+0x4695b1)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #23 std::__1::__function::__func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkudu_util.so+0x4688ad)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #24 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (kudu+0x5aa2b4)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #25 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (kudu+0x5aa0e9)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #26 kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x44a69a)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Mutex M3607 (0x7b5400001a58) created at:
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #0 pthread_rwlock_init /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1317 (kudu+0x4a842b)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #1 kudu::RWMutex::Init(kudu::RWMutex::Priority) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/rw_mutex.cc:78:8 (libkudu_util.so+0x4298d8)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #2 kudu::RWMutex::RWMutex(kudu::RWMutex::Priority) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/rw_mutex.cc:56:3 (libkudu_util.so+0x429b28)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #3 kudu::master::CatalogManager::CatalogManager(kudu::master::Master*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:987:7 (libmaster.so+0x317a0b)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #4 kudu::master::Master::Master(kudu::master::MasterOptions const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master.cc:268:28 (libmaster.so+0x400a0f)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #5 kudu::master::RunMasterServer() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/master_runner.cc:394:33 (libmaster.so+0x435ae3)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #6 kudu::tools::(anonymous namespace)::MasterRun(kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_action_master.cc:136:10 (kudu+0x5b84f9)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #7 decltype(std::__1::forward<kudu::Status (*&)(kudu::tools::RunnerContext const&)>(fp)(std::__1::forward<kudu::tools::RunnerContext const&>(fp0))) std::__1::__invoke<kudu::Status (*&)(kudu::tools::RunnerContext const&), kudu::tools::RunnerContext const&>(kudu::Status (*&)(kudu::tools::RunnerContext const&), kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (kudu+0x5355d1)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #8 kudu::Status std::__1::__invoke_void_return_wrapper<kudu::Status>::__call<kudu::Status (*&)(kudu::tools::RunnerContext const&), kudu::tools::RunnerContext const&>(kudu::Status (*&)(kudu::tools::RunnerContext const&), kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:317:16 (kudu+0x53553c)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #9 std::__1::__function::__alloc_func<kudu::Status (*)(kudu::tools::RunnerContext const&), std::__1::allocator<kudu::Status (*)(kudu::tools::RunnerContext const&)>, kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (kudu+0x5354ac)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #10 std::__1::__function::__func<kudu::Status (*)(kudu::tools::RunnerContext const&), std::__1::allocator<kudu::Status (*)(kudu::tools::RunnerContext const&)>, kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (kudu+0x534115)
09:07:20.529 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #11 std::__1::__function::__value_func<kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libkudu_tools_util.so+0x14045f)
09:07:20.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #12 std::__1::function<kudu::Status (kudu::tools::RunnerContext const&)>::operator()(kudu::tools::RunnerContext const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libkudu_tools_util.so+0x132cc1)
09:07:20.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #13 kudu::tools::Action::Run(std::__1::vector<kudu::tools::Mode*, std::__1::allocator<kudu::tools::Mode*> > const&, std::__1::unordered_map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::hash<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::equal_to<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > > const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_action.cc:245:10 (libkudu_tools_util.so+0x130a52)
09:07:20.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #14 kudu::tools::DispatchCommand(std::__1::vector<kudu::tools::Mode*, std::__1::allocator<kudu::tools::Mode*> > const&, kudu::tools::Action*, std::__1::deque<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_main.cc:142:15 (kudu+0x659330)
09:07:20.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #15 kudu::tools::RunTool(int, char**, bool) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_main.cc:214:16 (kudu+0x659c67)
09:07:20.530 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #16 main /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/tool_main.cc:309:10 (kudu+0x65a1ae)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Thread T78 'sq_worker' (tid=29738, running) created by thread T13 at:
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #0 pthread_create /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:966 (kudu+0x477c45)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #1 sq_start_thread /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:804:12 (libserver_process.so+0x1c43a0)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #2 try_start_another_worker /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:4449:7 (libserver_process.so+0x1c43a0)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #3 produce_socket /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:4464:5 (libserver_process.so+0x1c43a0)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #4 accept_new_connection /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:4521:5 (libserver_process.so+0x1c43a0)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #5 master_thread /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/squeasel-d83cf6d9af0e2c98c16467a6a035ae0d7ca21cb1/squeasel.c:4559:11 (libserver_process.so+0x1c43a0)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Thread T77 'leader-initiali' (tid=29737, running) created by thread T72 at:
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #0 pthread_create /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:966 (kudu+0x477c45)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #1 kudu::Thread::StartThread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:643:15 (libkudu_util.so+0x449f38)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #2 kudu::Thread::Create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, scoped_refptr<kudu::Thread>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.h:147:12 (kudu+0x59a529)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #3 kudu::ThreadPool::CreateThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:814:10 (libkudu_util.so+0x46657b)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #4 kudu::ThreadPool::DoSubmit(std::__1::function<void ()>, kudu::ThreadPoolToken*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:599:21 (libkudu_util.so+0x46481f)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #5 kudu::ThreadPool::Submit(std::__1::function<void ()>) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:491:10 (libkudu_util.so+0x466ae2)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #6 kudu::master::CatalogManager::ElectedAsLeaderCb() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1088:33 (libmaster.so+0x319139)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #7 kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:1680:40 (libmaster.so+0x355538)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #8 decltype(std::__1::forward<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15&>(fp)()) std::__1::__invoke<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15&>(kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libmaster.so+0x3554f0)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #9 kudu::Status std::__1::__invoke_void_return_wrapper<kudu::Status>::__call<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15&>(kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:317:16 (libmaster.so+0x355480)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #10 std::__1::__function::__alloc_func<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15, std::__1::allocator<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15>, kudu::Status ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libmaster.so+0x355438)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #11 std::__1::__function::__func<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15, std::__1::allocator<kudu::master::CatalogManager::InitSysCatalogAsync(bool)::$_15>, kudu::Status ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libmaster.so+0x354724)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #12 std::__1::__function::__value_func<kudu::Status ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libmaster.so+0x37c7fb)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #13 std::__1::function<kudu::Status ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libmaster.so+0x37c7a0)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #14 kudu::master::SysCatalogTable::SysCatalogStateChanged(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/sys_catalog.cc:461:16 (libmaster.so+0x479759)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #15 kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/sys_catalog.cc:498:15 (libmaster.so+0x481bdc)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #16 decltype(std::__1::forward<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0&>(fp)(std::__1::forward<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(fp0))) std::__1::__invoke<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libmaster.so+0x481b88)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #17 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&>(kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libmaster.so+0x481b08)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #18 std::__1::__function::__alloc_func<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0, std::__1::allocator<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0>, void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libmaster.so+0x481ac3)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #19 std::__1::__function::__func<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0, std::__1::allocator<kudu::master::SysCatalogTable::SetupTablet(scoped_refptr<kudu::tablet::TabletMetadata> const&)::$_0>, void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libmaster.so+0x480d6c)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #20 std::__1::__function::__value_func<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libtablet.so+0x3ed5b6)
09:07:20.531 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #21 std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)>::operator()(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libtablet.so+0x3ed548)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #22 kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/raft_consensus.cc:2857:3 (libconsensus.so+0x20421a)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #23 decltype(std::__1::forward<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16&>(fp)()) std::__1::__invoke<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16&>(kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libconsensus.so+0x2041c9)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #24 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16&>(kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libconsensus.so+0x204159)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #25 std::__1::__function::__alloc_func<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16, std::__1::allocator<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libconsensus.so+0x204121)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #26 std::__1::__function::__func<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16, std::__1::allocator<kudu::consensus::RaftConsensus::MarkDirty(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_16>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libconsensus.so+0x2033dd)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #27 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (kudu+0x5aa2b4)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #28 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (kudu+0x5aa0e9)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #29 kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7 (libkudu_util.so+0x467355)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #30 kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48 (libkudu_util.so+0x4696a1)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #31 decltype(std::__1::forward<kudu::ThreadPool::CreateThread()::$_1&>(fp)()) std::__1::__invoke<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkudu_util.so+0x469659)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #32 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libkudu_util.so+0x4695e9)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #33 std::__1::__function::__alloc_func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkudu_util.so+0x4695b1)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #34 std::__1::__function::__func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkudu_util.so+0x4688ad)
09:07:20.532 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #35 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (kudu+0x5aa2b4)
09:07:20.534 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #36 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (kudu+0x5aa0e9)
09:07:20.534 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) #37 kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x44a69a)
09:07:20.534 [INFO - cluster stderr printer] (MiniKuduCluster.java:751)
09:07:20.534 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) SUMMARY: ThreadSanitizer: data race /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:2490:19 in std::__1::unique_ptr<stack_st_X509, std::__1::function<void (stack_st_X509*)> >::get() const
09:07:20.534 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) ==================
09:07:20.535 [DEBUG - main] (Connection.java:748) [peer master-127.28.239.126:43219(127.28.239.126:43219)] connecting to peer
09:07:20.538 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.28.239.126:43219(127.28.239.126:43219)] Successfully connected to peer
09:07:20.591 [DEBUG - kudu-nio-0] (Negotiator.java:352) peer /127.28.239.126:43219 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.591 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.591528 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.538955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43706 (local address 127.28.239.126:43219)
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.539692 (+ 737us) server_negotiation.cc:207] Beginning negotiation
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.539718 (+ 26us) server_negotiation.cc:400] Waiting for connection header
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.540435 (+ 717us) server_negotiation.cc:408] Connection header received
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.541756 (+ 1321us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.541787 (+ 31us) server_negotiation.cc:462] Received NEGOTIATE request from client
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.542081 (+ 294us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.542255 (+ 174us) server_negotiation.cc:227] Negotiated authn=JWT
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.548924 (+ 6669us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.550081 (+ 1157us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.564130 (+ 14049us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.564631 (+ 501us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.565840 (+ 1209us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.589447 (+ 23607us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.590107 (+ 660us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43706: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":1190,"threads_started":1}
09:07:20.592 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:20.591835 29727 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43706: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.592 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.28.239.126:43219(127.28.239.126:43219)] handling channelInactive
09:07:20.593 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.28.239.126:43219(127.28.239.126:43219)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
09:07:20.594 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.28.239.126:43219: connection closed
09:07:20.594 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.28.239.126:43219; will retry
09:07:20.616 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.28.239.126:43219(127.28.239.126:43219)] connecting to peer
09:07:20.620 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.28.239.126:43219(127.28.239.126:43219)] Successfully connected to peer
09:07:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.678233 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
09:07:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.620394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43708 (local address 127.28.239.126:43219)
09:07:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.621146 (+ 752us) server_negotiation.cc:207] Beginning negotiation
09:07:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.621174 (+ 28us) server_negotiation.cc:400] Waiting for connection header
09:07:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.621948 (+ 774us) server_negotiation.cc:408] Connection header received
09:07:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.622803 (+ 855us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
09:07:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.622839 (+ 36us) server_negotiation.cc:462] Received NEGOTIATE request from client
09:07:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.623103 (+ 264us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
09:07:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.623236 (+ 133us) server_negotiation.cc:227] Negotiated authn=JWT
09:07:20.678 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.629417 (+ 6181us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.679 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.631015 (+ 1598us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
09:07:20.679 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.657927 (+ 26912us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.679 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.658542 (+ 615us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
09:07:20.679 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.661904 (+ 3362us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
09:07:20.679 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.672227 (+ 10323us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.679 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.672752 (+ 525us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43708: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.679 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":111}
09:07:20.681 [DEBUG - kudu-nio-1] (Negotiator.java:352) peer /127.28.239.126:43219 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.683 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.28.239.126:43219(127.28.239.126:43219)] handling channelInactive
09:07:20.683 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.28.239.126:43219(127.28.239.126:43219)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
09:07:20.683 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:20.683532 29727 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43708: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.684 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.28.239.126:43219: connection closed
09:07:20.685 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.28.239.126:43219; will retry
09:07:20.695 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.28.239.126:43219(127.28.239.126:43219)] connecting to peer
09:07:20.697 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.28.239.126:43219(127.28.239.126:43219)] Successfully connected to peer
09:07:20.728 [DEBUG - kudu-nio-0] (Negotiator.java:352) peer /127.28.239.126:43219 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.729 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.28.239.126:43219(127.28.239.126:43219)] handling channelInactive
09:07:20.729 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.729645 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.697943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.28.239.126:43219)
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.698449 (+ 506us) server_negotiation.cc:207] Beginning negotiation
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.698467 (+ 18us) server_negotiation.cc:400] Waiting for connection header
09:07:20.730 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.28.239.126:43219(127.28.239.126:43219)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.698504 (+ 37us) server_negotiation.cc:408] Connection header received
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.698836 (+ 332us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.698865 (+ 29us) server_negotiation.cc:462] Received NEGOTIATE request from client
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.699131 (+ 266us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.699296 (+ 165us) server_negotiation.cc:227] Negotiated authn=JWT
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.704100 (+ 4804us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.705661 (+ 1561us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.718275 (+ 12614us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.718871 (+ 596us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.719994 (+ 1123us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.727778 (+ 7784us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.728306 (+ 528us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43724: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.730 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":81}
09:07:20.731 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:20.730518 29727 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43724: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.731 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.28.239.126:43219: connection closed
09:07:20.731 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.28.239.126:43219; will retry
09:07:20.735 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.28.239.126:43219(127.28.239.126:43219)] connecting to peer
09:07:20.738 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.28.239.126:43219(127.28.239.126:43219)] Successfully connected to peer
09:07:20.764 [DEBUG - kudu-nio-1] (Negotiator.java:352) peer /127.28.239.126:43219 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.765 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.28.239.126:43219(127.28.239.126:43219)] handling channelInactive
09:07:20.765 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.28.239.126:43219(127.28.239.126:43219)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
09:07:20.765 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.765563 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
09:07:20.765 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.737841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43734 (local address 127.28.239.126:43219)
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.738340 (+ 499us) server_negotiation.cc:207] Beginning negotiation
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.738358 (+ 18us) server_negotiation.cc:400] Waiting for connection header
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.738504 (+ 146us) server_negotiation.cc:408] Connection header received
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.738971 (+ 467us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.738988 (+ 17us) server_negotiation.cc:462] Received NEGOTIATE request from client
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.739184 (+ 196us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.739303 (+ 119us) server_negotiation.cc:227] Negotiated authn=JWT
09:07:20.766 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.28.239.126:43219: connection closed
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.744360 (+ 5057us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.746006 (+ 1646us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.754366 (+ 8360us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.754952 (+ 586us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.755268 (+ 316us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.763786 (+ 8518us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.764321 (+ 535us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43734: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.766 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":90}
09:07:20.766 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.28.239.126:43219; will retry
09:07:20.767 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:20.765830 29727 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43734: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.776 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.28.239.126:43219(127.28.239.126:43219)] connecting to peer
09:07:20.777 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.28.239.126:43219(127.28.239.126:43219)] Successfully connected to peer
09:07:20.802 [DEBUG - kudu-nio-0] (Negotiator.java:352) peer /127.28.239.126:43219 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.803368 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
09:07:20.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.777846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43746 (local address 127.28.239.126:43219)
09:07:20.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.778345 (+ 499us) server_negotiation.cc:207] Beginning negotiation
09:07:20.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.778362 (+ 17us) server_negotiation.cc:400] Waiting for connection header
09:07:20.803 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.28.239.126:43219(127.28.239.126:43219)] handling channelInactive
09:07:20.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.778393 (+ 31us) server_negotiation.cc:408] Connection header received
09:07:20.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.778856 (+ 463us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
09:07:20.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.778873 (+ 17us) server_negotiation.cc:462] Received NEGOTIATE request from client
09:07:20.803 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.779032 (+ 159us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
09:07:20.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.779142 (+ 110us) server_negotiation.cc:227] Negotiated authn=JWT
09:07:20.804 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.28.239.126:43219(127.28.239.126:43219)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
09:07:20.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.784078 (+ 4936us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.785272 (+ 1194us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
09:07:20.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.795627 (+ 10355us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.796166 (+ 539us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
09:07:20.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.797370 (+ 1204us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
09:07:20.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.801810 (+ 4440us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.802190 (+ 380us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43746: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.804 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":73}
09:07:20.805 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:20.803632 29727 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43746: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.805 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.28.239.126:43219: connection closed
09:07:20.805 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.28.239.126:43219; will retry
09:07:20.836 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.28.239.126:43219(127.28.239.126:43219)] connecting to peer
09:07:20.837 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.28.239.126:43219(127.28.239.126:43219)] Successfully connected to peer
09:07:20.866 [DEBUG - kudu-nio-1] (Negotiator.java:352) peer /127.28.239.126:43219 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.867 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.866941 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
09:07:20.867 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.28.239.126:43219(127.28.239.126:43219)] handling channelInactive
09:07:20.867 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.837869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43756 (local address 127.28.239.126:43219)
09:07:20.867 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.838282 (+ 413us) server_negotiation.cc:207] Beginning negotiation
09:07:20.867 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.838297 (+ 15us) server_negotiation.cc:400] Waiting for connection header
09:07:20.867 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.838330 (+ 33us) server_negotiation.cc:408] Connection header received
09:07:20.867 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.838906 (+ 576us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
09:07:20.867 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.838931 (+ 25us) server_negotiation.cc:462] Received NEGOTIATE request from client
09:07:20.867 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.28.239.126:43219(127.28.239.126:43219)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
09:07:20.867 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.839145 (+ 214us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
09:07:20.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.839310 (+ 165us) server_negotiation.cc:227] Negotiated authn=JWT
09:07:20.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.844587 (+ 5277us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.846125 (+ 1538us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
09:07:20.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.855868 (+ 9743us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.856440 (+ 572us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
09:07:20.868 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.28.239.126:43219: connection closed
09:07:20.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.857564 (+ 1124us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
09:07:20.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.865200 (+ 7636us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.865732 (+ 532us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43756: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.868 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":72}
09:07:20.869 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:20.867899 29727 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43756: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.869 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.28.239.126:43219; will retry
09:07:20.915 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.28.239.126:43219(127.28.239.126:43219)] connecting to peer
09:07:20.917 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.28.239.126:43219(127.28.239.126:43219)] Successfully connected to peer
09:07:20.946 [DEBUG - kudu-nio-0] (Negotiator.java:352) peer /127.28.239.126:43219 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:20.946461 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
09:07:20.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.918176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43770 (local address 127.28.239.126:43219)
09:07:20.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.918785 (+ 609us) server_negotiation.cc:207] Beginning negotiation
09:07:20.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.918810 (+ 25us) server_negotiation.cc:400] Waiting for connection header
09:07:20.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.918855 (+ 45us) server_negotiation.cc:408] Connection header received
09:07:20.946 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.918997 (+ 142us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
09:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.919022 (+ 25us) server_negotiation.cc:462] Received NEGOTIATE request from client
09:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.919224 (+ 202us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
09:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.919382 (+ 158us) server_negotiation.cc:227] Negotiated authn=JWT
09:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.924365 (+ 4983us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.925927 (+ 1562us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
09:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.935594 (+ 9667us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.936159 (+ 565us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
09:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.937238 (+ 1079us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
09:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.944760 (+ 7522us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:20.945263 (+ 503us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43770: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":139}
09:07:20.947 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:20.947513 29727 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43770: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:20.947 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.28.239.126:43219(127.28.239.126:43219)] handling channelInactive
09:07:20.948 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.28.239.126:43219(127.28.239.126:43219)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
09:07:20.949 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.28.239.126:43219: connection closed
09:07:20.950 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.28.239.126:43219; will retry
09:07:21.035 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.28.239.126:43219(127.28.239.126:43219)] connecting to peer
09:07:21.037 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.28.239.126:43219(127.28.239.126:43219)] Successfully connected to peer
09:07:21.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:21.064323 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
09:07:21.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.037416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43786 (local address 127.28.239.126:43219)
09:07:21.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.037985 (+ 569us) server_negotiation.cc:207] Beginning negotiation
09:07:21.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.038009 (+ 24us) server_negotiation.cc:400] Waiting for connection header
09:07:21.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.038054 (+ 45us) server_negotiation.cc:408] Connection header received
09:07:21.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.038307 (+ 253us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
09:07:21.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.038338 (+ 31us) server_negotiation.cc:462] Received NEGOTIATE request from client
09:07:21.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.038556 (+ 218us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
09:07:21.064 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.038710 (+ 154us) server_negotiation.cc:227] Negotiated authn=JWT
09:07:21.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.043453 (+ 4743us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:21.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.045000 (+ 1547us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
09:07:21.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.054979 (+ 9979us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:21.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.055512 (+ 533us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
09:07:21.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.055792 (+ 280us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
09:07:21.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.063234 (+ 7442us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:21.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.063578 (+ 344us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43786: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:21.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":102}
09:07:21.065 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:21.064517 29727 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43786: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:21.066 [DEBUG - kudu-nio-1] (Negotiator.java:352) peer /127.28.239.126:43219 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:21.067 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.28.239.126:43219(127.28.239.126:43219)] handling channelInactive
09:07:21.068 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.28.239.126:43219(127.28.239.126:43219)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
09:07:21.068 [INFO - kudu-nio-1] (ConnectToCluster.java:385) Unable to connect to master 127.28.239.126:43219: connection closed
09:07:21.069 [WARN - kudu-nio-1] (ConnectToCluster.java:305) Unable to find the leader master 127.28.239.126:43219; will retry
09:07:21.175 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.28.239.126:43219(127.28.239.126:43219)] connecting to peer
09:07:21.177 [DEBUG - kudu-nio-0] (Connection.java:757) [peer master-127.28.239.126:43219(127.28.239.126:43219)] Successfully connected to peer
09:07:21.219 [DEBUG - kudu-nio-0] (Negotiator.java:352) peer /127.28.239.126:43219 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:21.223 [DEBUG - kudu-nio-0] (Connection.java:239) [peer master-127.28.239.126:43219(127.28.239.126:43219)] handling channelInactive
09:07:21.223 [DEBUG - kudu-nio-0] (Connection.java:709) [peer master-127.28.239.126:43219(127.28.239.126:43219)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
09:07:21.225 [INFO - kudu-nio-0] (ConnectToCluster.java:385) Unable to connect to master 127.28.239.126:43219: connection closed
09:07:21.226 [WARN - kudu-nio-0] (ConnectToCluster.java:305) Unable to find the leader master 127.28.239.126:43219; will retry
09:07:21.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:21.224995 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
09:07:21.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.177477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43798 (local address 127.28.239.126:43219)
09:07:21.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.178031 (+ 554us) server_negotiation.cc:207] Beginning negotiation
09:07:21.226 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.178053 (+ 22us) server_negotiation.cc:400] Waiting for connection header
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.178097 (+ 44us) server_negotiation.cc:408] Connection header received
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.178231 (+ 134us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.178254 (+ 23us) server_negotiation.cc:462] Received NEGOTIATE request from client
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.178452 (+ 198us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.184110 (+ 5658us) server_negotiation.cc:227] Negotiated authn=JWT
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.184469 (+ 359us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.185985 (+ 1516us) server_negotiation.cc:378] Sending TLS_HANDSHAKE NegotiatePB response
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.203180 (+ 17195us) server_negotiation.cc:366] Received TLS_HANDSHAKE NegotiatePB request
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.203724 (+ 544us) server_negotiation.cc:663] Negotiated TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.204828 (+ 1104us) server_negotiation.cc:366] Received JWT_EXCHANGE NegotiatePB request
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.212266 (+ 7438us) server_negotiation.cc:395] Sending RPC error: FATAL_INVALID_JWT: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 0204 09:07:21.212741 (+ 475us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43798: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) Metrics: {"server-negotiator.queue_time_us":111}
09:07:21.227 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) W20260204 09:07:21.226805 29727 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43798: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:21.516 [DEBUG - pool-22-thread-1] (Connection.java:748) [peer master-127.28.239.126:43219(127.28.239.126:43219)] connecting to peer
09:07:21.517 [DEBUG - kudu-nio-1] (Connection.java:757) [peer master-127.28.239.126:43219(127.28.239.126:43219)] Successfully connected to peer
09:07:21.536 [INFO - pool-22-thread-1] (ConnectToCluster.java:385) Unable to connect to master 127.28.239.126:43219: cannot complete before timeout: KuduRpc(method=ConnectToMaster, tablet=null, attempt=1, TimeoutTracker(timeout=19, elapsed=19), Traces: [0ms] sending RPC to server master-127.28.239.126:43219, deferred=Deferred@487862075(state=PENDING, result=null, callback=passthrough -> ConnectToMasterCB for 127.28.239.126:43219, errback=org.apache.kudu.client.ConnectToCluster$1@1fadc445 -> ConnectToMasterErrCB for 127.28.239.126:43219))
09:07:21.538 [DEBUG - pool-22-thread-1] (AsyncKuduClient.java:1523) Notify RPC KuduRpc(method=GetTableSchema, tablet=null, attempt=10, TimeoutTracker(timeout=1000, elapsed=1002), Traces: [0ms] refreshing cache from master, [1ms] Sub RPC ConnectToMaster: sending RPC to server master-127.28.239.126:43219, [59ms] Sub RPC ConnectToMaster: received response from server master-127.28.239.126:43219: Network error: connection closed, [60ms] delaying RPC due to: Service unavailable: Master config (127.28.239.126:43219) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [81ms] refreshing cache from master, [82ms] Sub RPC ConnectToMaster: sending RPC to server master-127.28.239.126:43219, [149ms] Sub RPC ConnectToMaster: received response from server master-127.28.239.126:43219: Network error: connection closed, [152ms] delaying RPC due to: Service unavailable: Master config (127.28.239.126:43219) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [161ms] refreshing cache from master, [161ms] Sub RPC ConnectToMaster: sending RPC to server master-127.28.239.126:43219, [196ms] Sub RPC ConnectToMaster: received response from server master-127.28.239.126:43219: Network error: connection closed, [198ms] delaying RPC due to: Service unavailable: Master config (127.28.239.126:43219) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [201ms] refreshing cache from master, [201ms] Sub RPC ConnectToMaster: sending RPC to server master-127.28.239.126:43219, [231ms] Sub RPC ConnectToMaster: received response from server master-127.28.239.126:43219: Network error: connection closed, [233ms] delaying RPC due to: Service unavailable: Master config (127.28.239.126:43219) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [241ms] refreshing cache from master, [241ms] Sub RPC ConnectToMaster: sending RPC to server master-127.28.239.126:43219, [270ms] Sub RPC ConnectToMaster: received response from server master-127.28.239.126:43219: Network error: connection closed, [271ms] delaying RPC due to: Service unavailable: Master config (127.28.239.126:43219) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [301ms] refreshing cache from master, [302ms] Sub RPC ConnectToMaster: sending RPC to server master-127.28.239.126:43219, [333ms] Sub RPC ConnectToMaster: received response from server master-127.28.239.126:43219: Network error: connection closed, [335ms] delaying RPC due to: Service unavailable: Master config (127.28.239.126:43219) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [381ms] refreshing cache from master, [381ms] Sub RPC ConnectToMaster: sending RPC to server master-127.28.239.126:43219, [414ms] Sub RPC ConnectToMaster: received response from server master-127.28.239.126:43219: Network error: connection closed, [417ms] delaying RPC due to: Service unavailable: Master config (127.28.239.126:43219) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [501ms] refreshing cache from master, [501ms] Sub RPC ConnectToMaster: sending RPC to server master-127.28.239.126:43219, [534ms] Sub RPC ConnectToMaster: received response from server master-127.28.239.126:43219: Network error: connection closed, [535ms] delaying RPC due to: Service unavailable: Master config (127.28.239.126:43219) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [641ms] refreshing cache from master, [641ms] Sub RPC ConnectToMaster: sending RPC to server master-127.28.239.126:43219, [689ms] Sub RPC ConnectToMaster: received response from server master-127.28.239.126:43219: Network error: connection closed, [693ms] delaying RPC due to: Service unavailable: Master config (127.28.239.126:43219) has no leader. Exceptions received: org.apache.kudu.client.RecoverableException: connection closed, [981ms] refreshing cache from master, [981ms] Sub RPC ConnectToMaster: sending RPC to server master-127.28.239.126:43219, deferred=Deferred@183759617(state=PENDING, result=null, callback=(continuation of Deferred@1383964606 after retry RPC after error@1848385341) -> (continuation of Deferred@533881169 after retry RPC after error@170713681) -> (continuation of Deferred@881986757 after retry RPC after error@986261661) -> (continuation of Deferred@907873916 after retry RPC after error@1531858052) -> (continuation of Deferred@2132707084 after retry RPC after error@207668658) -> (continuation of Deferred@1380996268 after retry RPC after error@2077427726) -> (continuation of Deferred@1887356575 after retry RPC after error@2058993783) -> (continuation of Deferred@706481848 after retry RPC after error@1231872141) -> (continuation of Deferred@1327533313 after retry RPC after error@2066634474), errback=(continuation of Deferred@1383964606 after retry RPC after error@1848385341) -> (continuation of Deferred@533881169 after retry RPC after error@170713681) -> (continuation of Deferred@881986757 after retry RPC after error@986261661) -> (continuation of Deferred@907873916 after retry RPC after error@1531858052) -> (continuation of Deferred@2132707084 after retry RPC after error@207668658) -> (continuation of Deferred@1380996268 after retry RPC after error@2077427726) -> (continuation of Deferred@1887356575 after retry RPC after error@2058993783) -> (continuation of Deferred@706481848 after retry RPC after error@1231872141) -> (continuation of Deferred@1327533313 after retry RPC after error@2066634474))) after lookup exception
org.apache.kudu.client.NonRecoverableException: Couldn't find a valid master in (127.28.239.126:43219). Exceptions received: [org.apache.kudu.client.NonRecoverableException: cannot complete before timeout: KuduRpc(method=ConnectToMaster, tablet=null, attempt=1, TimeoutTracker(timeout=19, elapsed=19), Traces: [0ms] sending RPC to server master-127.28.239.126:43219, deferred=Deferred@487862075(state=PENDING, result=null, callback=passthrough -> ConnectToMasterCB for 127.28.239.126:43219, errback=org.apache.kudu.client.ConnectToCluster$1@1fadc445 -> ConnectToMasterErrCB for 127.28.239.126:43219))]
at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:268) [main/:?]
at org.apache.kudu.client.ConnectToCluster.access$000(ConnectToCluster.java:51) [main/:?]
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:387) [main/:?]
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:376) [main/:?]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280) [async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) [async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315) [async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286) [async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259) [async-1.4.1.jar:?]
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002) [async-1.4.1.jar:?]
at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:277) [main/:?]
at org.apache.kudu.client.KuduRpc.errback(KuduRpc.java:331) [main/:?]
at org.apache.kudu.client.KuduRpc$RpcTimeoutTask.run(KuduRpc.java:466) [main/:?]
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:713) [netty-common-4.1.115.Final.jar:4.1.115.Final]
at io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) [netty-common-4.1.115.Final.jar:4.1.115.Final]
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:701) [netty-common-4.1.115.Final.jar:4.1.115.Final]
at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:788) [netty-common-4.1.115.Final.jar:4.1.115.Final]
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:501) [netty-common-4.1.115.Final.jar:4.1.115.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
org.apache.kudu.client.NonRecoverableException: Couldn't find a valid master in (127.28.239.126:43219). Exceptions received: [org.apache.kudu.client.NonRecoverableException: cannot complete before timeout: KuduRpc(method=ConnectToMaster, tablet=null, attempt=1, TimeoutTracker(timeout=19, elapsed=19), Traces: [0ms] sending RPC to server master-127.28.239.126:43219, deferred=Deferred@487862075(state=PENDING, result=null, callback=passthrough -> ConnectToMasterCB for 127.28.239.126:43219, errback=org.apache.kudu.client.ConnectToCluster$1@1fadc445 -> ConnectToMasterErrCB for 127.28.239.126:43219))]
at org.apache.kudu.client.ConnectToCluster.incrementCountAndCheckExhausted(ConnectToCluster.java:268)
at org.apache.kudu.client.ConnectToCluster.access$000(ConnectToCluster.java:51)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:387)
at org.apache.kudu.client.ConnectToCluster$ConnectToMasterErrCB.call(ConnectToCluster.java:376)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1280)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.handleContinuation(Deferred.java:1315)
at com.stumbleupon.async.Deferred.doCall(Deferred.java:1286)
at com.stumbleupon.async.Deferred.runCallbacks(Deferred.java:1259)
at com.stumbleupon.async.Deferred.callback(Deferred.java:1002)
at org.apache.kudu.client.KuduRpc.handleCallback(KuduRpc.java:277)
at org.apache.kudu.client.KuduRpc.errback(KuduRpc.java:331)
at org.apache.kudu.client.KuduRpc$RpcTimeoutTask.run(KuduRpc.java:466)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:713)
at io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34)
at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:701)
at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:788)
at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:501)
at java.lang.Thread.run(Thread.java:748)
09:07:21.540 [DEBUG - main] (AsyncKuduClient.java:2704) Releasing all remaining resources
09:07:21.541 [DEBUG - kudu-nio-1] (Negotiator.java:352) peer /127.28.239.126:43219 sent connection negotiation error: Not authorized: JWT verification failed: failed to verify signature: VerifyFinal failed (error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding:../crypto/rsa/rsa_pk1.c:67 error:04067072:rsa routines:rsa_ossl_public_decrypt:padding check failed:../crypto/rsa/rsa_ossl.c:581)
09:07:21.542 [DEBUG - kudu-nio-1] (Connection.java:239) [peer master-127.28.239.126:43219(127.28.239.126:43219)] handling channelInactive
09:07:21.542 [DEBUG - kudu-nio-1] (Connection.java:709) [peer master-127.28.239.126:43219(127.28.239.126:43219)] cleaning up while in state NEGOTIATION_FAILED due to: connection closed
09:07:21.544 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) I20260204 09:07:21.543622 29629 external_mini_cluster.cc:1658] Killing /tmp/dist-test-task5fNXJ8/build/tsan/bin/kudu with pid 29656
09:07:21.545 [DEBUG - kudu-nio-1] (KuduRpc.java:260) Handling a callback on RPC KuduRpc(method=ConnectToMaster, tablet=null, attempt=0, TimeoutTracker(timeout=0, elapsed=9), Traces: [0ms] received response from server master-127.28.239.126:43219: Network error: connection closed, deferred=null) with no deferred attached!
09:07:22.613 [INFO - cluster stderr printer] (MiniKuduCluster.java:751) 2026-02-04T09:07:22Z chronyd exiting
09:07:23.649 [ERROR - main] (RetryRule.java:206) org.apache.kudu.client.TestNegotiation.testInvalidJwt: failed attempt 1
junit.framework.AssertionFailedError: null
at junit.framework.Assert.fail(Assert.java:55) ~[junit-4.13.2.jar:4.13.2]
at junit.framework.Assert.assertTrue(Assert.java:22) ~[junit-4.13.2.jar:4.13.2]
at junit.framework.Assert.assertTrue(Assert.java:31) ~[junit-4.13.2.jar:4.13.2]
at junit.framework.TestCase.assertTrue(TestCase.java:200) ~[junit-4.13.2.jar:4.13.2]
at org.apache.kudu.client.TestNegotiation.testInvalidJwt(TestNegotiation.java:280) ~[test/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_275]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_275]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_275]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_275]
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) ~[junit-4.13.2.jar:4.13.2]
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) ~[junit-4.13.2.jar:4.13.2]
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) ~[junit-4.13.2.jar:4.13.2]
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) ~[junit-4.13.2.jar:4.13.2]
at org.apache.kudu.test.junit.RetryRule$RetryStatement.doOneAttemptAndReport(RetryRule.java:184) [kudu-test-utils-1.19.0-SNAPSHOT-unshaded.jar:1.19.0-SNAPSHOT]
at org.apache.kudu.test.junit.RetryRule$RetryStatement.evaluate(RetryRule.java:244) [kudu-test-utils-1.19.0-SNAPSHOT-unshaded.jar:1.19.0-SNAPSHOT]
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner.run(ParentRunner.java:413) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.Suite.runChild(Suite.java:128) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.Suite.runChild(Suite.java:27) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) [junit-4.13.2.jar:4.13.2]
at org.junit.runners.ParentRunner.run(ParentRunner.java:413) [junit-4.13.2.jar:4.13.2]
at org.junit.runner.JUnitCore.run(JUnitCore.java:137) [junit-4.13.2.jar:4.13.2]
at org.junit.runner.JUnitCore.run(JUnitCore.java:115) [junit-4.13.2.jar:4.13.2]
at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77) [junit-4.13.2.jar:4.13.2]
at org.junit.runner.JUnitCore.main(JUnitCore.java:36) [junit-4.13.2.jar:4.13.2]
junit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:55)
at junit.framework.Assert.assertTrue(Assert.java:22)
at junit.framework.Assert.assertTrue(Assert.java:31)
at junit.framework.TestCase.assertTrue(TestCase.java:200)
at org.apache.kudu.client.TestNegotiation.testInvalidJwt(TestNegotiation.java:280)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at org.apache.kudu.test.junit.RetryRule$RetryStatement.doOneAttemptAndReport(RetryRule.java:184)
at org.apache.kudu.test.junit.RetryRule$RetryStatement.evaluate(RetryRule.java:244)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
at org.junit.runner.JUnitCore.runMain(JUnitCore.java:77)
at org.junit.runner.JUnitCore.main(JUnitCore.java:36)